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the aggregate the elements of the legal memoranda that were most important 
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aggregate what elements of the legal memoranda were most important and what relative weight each has in 
the judgments of the instructors and professors. It was also possible to identify combinations of elements that 
formed factors important in determining the quality of legal memoranda. Computer analyses of the 
memoranda indicated differences between the student memoranda and other kinds of writing, particularly with 
respect to the greater use of passive voice and nominalizations in the student memoranda, but the computer 
variables did not correlate significantly with global quality ratings of the memoranda. Although no significant 
sex differences were found in the global quality ratings, differences favoring females were observed for several 
taxonomy elements. 
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INTRODUCTION 

Although legal writing has obviously been around for a long time, law schools have not until 
relatively recently taught courses in legal writing. Legal writing did not appear as a category in the 
American Association of Law Schools’ Directory of Teachers in Member Schools until 1947, and less than 
50 years have passed since writing courses were included in the law school curriculum (Carrick & Dunn, 
1985). Moreover, the earliest courses were remedial courses focussing on basic writing skills (Rombauer, 
1973). 

The Growing Interest in Legal Writing 

If publications are representative of interest in a field, then interest in legal writing presently is 
strong and growing. Gopen and Smout (1990) listed 95 books and 361 articles on the topic published 
between 1960 and 1990. Of the books, 22 percent were published before 1970, 21 percent in the 1970s, 
and 57 percent in the 1980s. Of the articles, 19 percent were published before 1960, 14 percent in the 
1960s, 23 percent in the 1970s, and 44 percent in the 1980s. Clearly, the 1980s was a period of great 
growth in the profession of legal writing as evidenced by publications. The interest appears to be 
continuing into the 1990s. For example, one leading law school publisher introduced two new books on 
legal writing in 1990 (Neumann, 1990; Calleros, 1990). At least three new journals have appeared: The 
American Society of Writers on Legal Subjects (Scribes) began publishing The Scribes Journal of Legal 
Writing (first published in 1990), the Legal Writing Institute (LWI) began publishing The Journal of the 
Legal Writing Institute (first published in 1991), and West Publishing Co. began publishing Perspectives: 
Teaching Legal Research and Writing (first published in 1992). Finally, "substantive" law reviews are 
now beginning to publish articles in the field of legal writing (see, e.g., Fajans & Falk, 1993, an article 
about critical reading theory and upper-year writing courses). 

The increase in publications may be related to increasing concerns among groups of teachers of 
legal writing, such as the Legal Writing Institute. The Legal Writing Institute was established in 1984 
after manifestations of unrest because of high turnover among legal research and writing professionals, 
low pay, and difficulties in obtaining tenured status in law schools. Courses were often graded pass-fail 
and course credit units were low, resulting in low student interest (Shapo, 1986). Turnover in the 
profession has always been and still is high. Fifty-five percent of full-time professionals made less than 
$30,000 per year in 1990 (Ramsfield, 1991). The Legal Writing Institute’s purpose is to unite legal 
research and writing professionals intellectually, to share resources, and to monitor and encourage the 
development of effective courses across the United States and Canada. 

Criticism of Legal Writing 

The increased interest in legal writing in recent years has been stimulated by numerous complaints 
about the quality of legal writing (see, e.g., Besharov & Hartle, 1985; Gale, 1980; Graham, 1983; Hazard, 
1982). Some criticisms are attacks on "legalese," and call for the use of "plain English." Other criticisms 
are broader in scope. In the late 1970s, the Committee of Bar Examiners of the State of California came 
under attack by partners in the large law firms across the state. These partners had observed that law 
graduates who had passed the bar exam "could not prepare a memo that accurately summarized how the 
findings from a search of the legal literature related to their client’s case." (Klein, 1989). Since these 
firms did not hire marginal graduates, but only the most able, it was clear that the problem was a 
pervasive one and not limited to basic English skills. A recent study of the relationship between legal 
education and the practice of law, listed "communication" as a necessary legal skill. The study concluded 
that, among other things, a lawyer must be able to organize a presentation, express ideas with precision, 
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clarity, logic and economy, use appropriate terminology, apply the mechanics of language, and attend to 
detail (American Bar Association, 1992, p. 173). 

Books on Legal Writing 

One way to trace the development of legal writing as a profession is to examine textbooks used 
in legal writing courses over the years. Carrick and Dunn (1985) conducted such an examination over 
the years up until the time of their article. They note that Cooper’s Effective Legal Writing (Cooper, 
1953) was one of the first books devoted solely to legal writing. This book assumed a reasonable 
knowledge of English grammar and focussed on the drafting of various types of legal documents. 
Although some in the field viewed this book as the best available source of instruction in legal writing, 
others found fault with it and it was never widely accepted. 

The first book to receive widespread acceptance was Weihofen’s Legal Writing Style (Wiehofen, 
1961). This book instructed in composition in ordinary written English and then addressed specific types 
of legal writing including letters, opinions, memoranda, and briefs. It became the assigned text in about 
seventy law schools, and a second edition was published twenty years later (Wiehofen, 1980). Carrick 
and Dunn describe the book as practical, with specific rules to follow for clear, strong writing. Another 
book of about the same era. Brand and White’s Legal Writing: The Strategy of Persuasion (Brand & 
White, 1976), was also intended primarily to help students express ideas clearly and effectively. 

Although of the same period as the Wiehofen and Brand and White texts, a course book authored 
by Rombauer, Legal Analysis and Research (Rombauer, 1970), suggested a fundamental change in 
approach to instruction in legal writing. As the title indicates, legal analysis and research are the 
emphasis; writing is not mentioned at all. The second edition of this book, published in 1973, was 
entitled, Legal Problem Solving: Analysis „ Research, and Writing, and has retained that title in two 
subsequent editions of this book (Rombauer, 1978, 1983). The emphasis on problem solving and analysis, 
and the subordination of research and writing, signalled a departure from the basic skills approach that 
had been prominent in legal writing up until that time. Another book of this type was Statsky and 
Wemet’s Case Analysis and Fundamentals of Legal Writing (Statsky & Wemet, 1977, 1984), which also 
emphasized problem solving; it was the first book to integrate legal cases into a text for legal writing and 
analysis. 

A number of books published since 1980 also integrate legal writing with legal analysis, but some 
still emphasize primarily grammar, style, and mechanics. Block’s Effective Legal Writing: A Style Book 
for Law Students and Lawyers (Block, 1981) is described by Carrick and Dunn (1985) as a "technical 
study of grammar with rules, guidelines, and exercises," although it includes a brief discussion of the 
IRAC (issue, rule, analysis, and conclusion) method. Dembach and Singleton’s A Practical Guide to 
Legal Writing (Dembach & Singleton, 1981), emphasizes practice in case and statutory analysis and trains 
the student to identify facts, holdings, and issues. Many exercises are included. Gopen’s Writing from 
a Legal Perspective (Gopen, 1981) stresses basic writing problems (grammar and style) in the first half 
and covers the types of writing that lawyers do in the second half. Mellinkoffs Legal Writing: Sense and 
Nonsense (Mellinkoff, 1982) is a critical review of legal writing and is typical of scathing portrayals of 
current legal writing, according to Carrick and Dunn, and emphasizes precision, clarity, and conciseness. 
Squires and Rombauer’s Legal Writing in a Nutshell (Squires & Rombauer, 1982) stresses organization, 
sentence structure, grammar, and punctuation. It includes a glossary of words often misused, and covers 
various types of legal writing. 
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Carrick and Dunn conclude their 1985 review by observing that many of the books overemphasize 
basic grammar, topics that are handled better in English composition books. None of the books they 
reviewed effectively combined legal research with legal writing. In their view, Dembach and Singleton 
(1981) and Mellinkoff (1982) best balanced grammar and rules of composition with their practical 
application to legal situations and thus represented the best texts available at that time (1985) for basic 
legal writing courses. 

Other books on legal writing have appeared since the Carrick and Dunn review. Calleros’s Legal 
Method in Writing begins with four chapters devoted to legal reasoning before tackling writing style 
(Calleros, 1990). Neumann’s Legal Reasoning and Legal Writing: Structure, Strategy, and Style contains 
substantial material on working with rules, skills for using precedent, and statutory analysis (Neumann, 
1990). Shapo et al’s Writing and Analysis in the Law (Shapo et al., 1991) contains thirteen chapters and 
five appendixes; only three chapters and one appendix deal primarily with composition and grammar. 
Pratt’s Legal Writing: A Systematic Approach (Pratt, 1990) also emphasizes legal topics; very little 
attention is given to standard English composition rules, although there is some discussion of 
active/passive voice. Tepley’s Legal Writing, Analysis, and Oral Argument (Tepley, 1990) belongs in this 
group as well because it is mostly about legal analysis and research, although it is described as a "self- 
teaching text," rather than as a coursebook. 

Other coursebooks published since 1985 have a distinctly different nature. Charrow and Erhardt’s 
Clear and Effective Legal Writing (Charrow & Erhardt, 1986) contains much about clarity, sentence 
structure, audience, the writing process, etc.; there is no mention of legal analysis per se, even though 
there is some brief discussion of application of law to facts, rule identification, and consideration of 
counterarguments. Porter et al.’s Introduction to Legal Writing and Oral Advocacy begins by describing 
the various types of legal writing and then concentrates on organization, sentence structure, active voice, 
quotations, punctuation, etc. There is little discussion of legal analysis. Gilmer’s Legal Research, 
Writing, and Advocacy: A Sourcebook for Students, Lawyers, and Paralegals (Gilmer, 1987) is unique 
with its heavy emphasis on research and library work, nothing on English composition, and little on legal 
analysis. The book probably reflects Gilmer’s many years of work as a law librarian. 

A final group of recent books on legal writing are distinguished in part because they are not 
coursebooks and in part because they tend to emphasize English composition more than legal topics. Ray 
and Ramsfield’s Legal Writing: Getting It Right and Getting It Written (Ray & Ramsfield, 1987) is a 
desktop reference for legal writers designed to be a companion to the Harvard Uniform System of Citation. 
The alphabetical list of topics contains some legal terms, which are described, and some topics on the 
writing process (such as how to get started with a legal writing task), but most of the topics are standard 
English composition topics. Katz’s Winning Words: A Guide to Persuasive Writing for Lawyers (Katz, 
1986) begins with a discussion of style and proceeds to emphasize brevity, simplicity, sentence structure, 
grammar, and active voice. It contains many examples of the various types of legal writing, and some 
discussion of analysis, but nothing on the structure of the law, the courts, etc. Good’s Mightier Than the 
Sword: Powerful Writing in the Legal Profession (Good, 1989) represents an extreme among the more 
recent books in that it is entirely about English composition. 

Of the books published since Carrick and Dunn’s 1985 review, the coursebooks appear to continue 
a trend toward more legal analysis and less English composition. No such trend is apparent in other than 
coursebooks. Few books of any type, however, accept Carrick and Dunn’s suggestion that English 
composition is best left to English books. 
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Law School Admissions and Writing Skill 

The Law School Admission Test (LSAT) has long addressed the question of applicants’ writing 
skills. In the early 1950s a study was conducted of possible tests of legal writing ability (Coffman & 
Papachristou, 1955) and in 1954 five items designed to measure writing ability were administered on an 
experimental basis (Olsen, 1956), and the value of writing items as a predictor of law school grades was 
first investigated in 1962. The motivations for introducing a test of writing ability were (Pitcher, 1962, 
p. 204): 

(1) Since the ability to write is very important to the law student and to the practicing lawyer, 
some evidence of proficiency in writing might be useful in helping to determine the 
admissibility of law school applicants. 

(2) The presence of a test of writing ability in the LSAT battery might help to encourage pre- 
law students to prepare themselves in writing during their undergraduate careers. 

A machine-scoreable writing ability test was introduced in 1962. It was scored and reported 
separately on a 20 to 80 scale. Pitcher’s prediction that the writing ability items would not add much to 
prediction of first year grades when combined with the LSAT and undergraduate grade-point average 
proved to be correct (Pitcher, 1965). Some schools used the writing ability score as a third predictor in 
the regression equation predicting law school grades, but it never added much, and fluctuations were 
common (Schrader, 1976). Some believed that the writing ability score was of assistance in the 
assessment of minority candidates for law school, but a study showed that this was not true (Powers, 
1980). 



In 1980, the Law School Admission Council Board of Trustees voted to eliminate the writing 
ability items and the writing ability score. At the same time, the Board voted to include an unscored 
written writing sample. Two reasons for the inclusion of the writing sample were given: (1) it would 
inform candidates that law schools believe writing to be important, and (2) it would provide a means for 
allowing law schools to make an independent judgment as to an applicant’s writing skills (Law School 
Admission Council, 1980, p. 61). 

The writing sample was introduced in 1982, and it has continued as a part of the administration 
ever since. Two issues have recurred in connection with the writing sample: (1) the proper format of the 
stimulus and (2) whether it should be scored. The first issue was addressed in 1985, and is again under 
consideration. The second issue has been debated periodically, with those who favor scoring arguing that 
the writing sample is little used without scoring, and those opposed to scoring arguing that the score would 
not add to prediction of law school grades and would, therefore, be misleading. 

One of the areas selected for research by the LSAT Test Development Committee is legal writing. 
The ultimate aim is to develop a method of determining with some degree of precision the ability of law 
school applicants to succeed in the many aspects of legal training that require writing. Initially, the 
question arises as to what is meant by "legal writing." 

The Nature of Legal Writing 

Early discussions among members of the Law School Admission Council, Law School Admission 
Services, and Educational Testing Service focussed on this question of the nature of legal writing. How 
does it differ from other kinds of writing? Is there something unique in "thinking like a lawyer" that 
carries over into legal writing and makes it unique? There seemed to be no clear answer to such 
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questions, because there appeared to be considerable disagreement within the legal profession about just 
what constitutes legal writing. Consider the following statements: 

A thorough understanding of the legal problem-solving process is of little value 
. . . unless the analysis can be communicated on paper. Good legal writing is 
essentially the same as good writing in general. 

[Dembach and Singleton (1981), p. xvii] 

Basically, the kind of writing required for (legal) assignments is just a variation 
of persuasive prose. The skill to write law examinations, memoranda, and briefs, 
since the basics are not peculiar to law, can be taught— and can be taught even by 
those who are not lawyers. 

[White & Brand (1976), p. 41] 

The ability to write an organized, persuasive argument is in no way peculiar or 
special to the legal profession. It follows that law teachers are no better and 
indeed perhaps less equipped to teach writing skills than would be those persons 
in the university who approach that task as their specialty. 

[Pedrick, in Pedrick et al. (1982), p. 414.] 

Compare the above statements with these: 

Of all the current verities in the legal writing instructors’ lounge, none is 
embraced more passionately than the idea that by teaching legal writing one is 
teaching, and improving, fundamental "legal thinking" skills and processes. 

Where, legal writing teachers will argue, does genuine legal thinking and analysis 
occur more intensively than during the research and writing process? 

[Stratman, 1990, p. 153] 

The problem with legal writing is not that there are too many "herein-befores" and 
not enough metaphors. The problem is that lawyers cannot write clearly unless 
they can think clearly, unless they can recognize and construct a convincing legal 
argument— unless, in other words, they understand the structure of the law. 

[Hyland, 1986, p. 621, quoted in Gopen, 1987, p. 353] 

Legal reasoning and legal writing are taught together in this text because the 
reasoning and writing processes are so inseparable that one cannot fully be taught 
without the other. For several years now, legal writing teachers have known that 
writing is better learned when combined with at least some instruction in legal 
reasoning. [Neumann, 1990, p. xix] 

[Techniques of expression are closely linked to the underlying substantive 
analysis; indeed, problems in writing style often betray confusion in the analysis. 

[Calleros, 1990, p. xxiii] 

Evolution of the Project 

The general consensus that there was disagreement in the law teaching profession about just what 
constitutes legal writing and how it differs or does not differ from other types of writing, led to the idea 
of conducting a project that would begin to define what legal writing entails. The project desired would 
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preferably be objective and empirical because of the disagreement within the legal profession. Objectivity 
would require project direction outside of the legal profession. An Advisory Committee of legal writing 
experts would be needed, however, to guide the project, with a liaison from LSAC to attend meetings and 
thus reflect LSAC interests. 

In considering ways to approach the problem empirically, it seemed advisable to consider past 
efforts where attempts had been made to define other types of writing empirically. There had been a 
significant research investigation at Educational Testing Service in the early 1960s in which a large 
amount of data had been collected and analyzed to examine schools of thought in judging excellence in 
English themes. This study, Diederich, French, and Carlton (1961) began with 300 brief essays written 
as homework by college freshmen. To obtain a wide range of judgment on the quality and specific 
characteristics of these essays, readers were selected from a assortment of fields of specialty: English 
teachers, social scientists, natural scientists, writers or editors, lawyers, and business executives. These 
readers were asked to use their own judgment about what constitutes "writing ability." They were 
instructed to sort the 300 essays into nine different piles of merit and, additionally, to write comments on 
the papers as to why they liked or disliked them. The results of this exercise revealed much disagreement 
among the readers as to which essays were the best and worst and what specifically was good or bad 
about them. No essay received less than five different grades, and 101 of the 300 essays received all nine 
different grades. 

This study of English essays, published later in a condensed form by French (1962), examined 
50,000 comments totalling 155,000 words to identify 55 categories of comments. Despite disagreements 
among the various judges, it was still possible to derive meaning from the data. To do so, the 55 
categories of comments were subjected to a statistical technique called factor analysis to.determine which 
tended to go together to form what might be called schools of thought, transcending the various 
professions, as to what English composition ability is. The results revealed five factors labelled as 
follows: IDEAS, FORM, FLAVOR, MECHANICS, and WORDING. IDEAS was related to quantity of ideas 
and clarity of ideas, FORM to organization and coherence, FLAVOR to style and originality, MECHANICS 
to punctuation, grammar, and sentence structure, and WORDING to word choices, cliches, and slang. 

After discussion of this study with representatives of the Law School Admission Council, Law 
School Admission Services, and Educational Testing Service researchers, it was decided that a similar 
study of legal writing would be useful. The objective of this study would be to determine what factors 
legal writing instructors believe important in good legal writing. The approach would be to have law 
school teachers of writing "grade" papers prepared by students in actual classes. The overall grade given 
to a paper would be of less importance than the comments instructors wrote on the papers during the 
grading process. Through these comments, refined with the assistance of the Advisory Committee, a 
taxonomy of legal writing could be developed. The taxonomy, in effect, discloses those qualities of law 
school writing assignments that were of importance to the graders. By having the graders rate the extent 
to which a paper successfully demonstrated a particular quality, it would be possible to draw some 
conclusions about the importance of that quality in comparison with other qualities. 

Legal writing assignments in law school vary significantly. Students may have to write legal 
memoranda, appellate briefs, letters to clients or to opposing lawyers, legislation, contracts, wills, etc. It 
is at least arguable that different skills are required for each type of legal writing. There is also a 
difference in the expertise expected of students depending upon the amount of legal education that they 
have had at the time the writing assignment is undertaken. It was decided that first semester legal 
memoranda would be the focus of this study. Most, if not all, law schools have a legal writing course 
or program in the first semester, and most use the legal memorandum for one or more of the exercises. 
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DESCRIPTION OF THE STUDY 

The Advisory Committee was formed in the summer and early fall of 1991, and the first meeting 
was held in October. It was decided at that meeting that from 10 to 15 law schools should participate in 
a study of student samples of legal memoranda written during the first semester of law school. Each law 
school would be asked to supply twenty writing samples representing a range of the abilities of students 
in each law school. The possibility of using a common assignment was considered but rejected because 
it would not provide a range of assignments and because it would require long-range planning. In 
rejecting the use of a common question, it was recognized that agreement between graders of the same 
paper would suffer because it is more likely that graders would have a more common framework if they 
were all rating papers responding to the same assignment. However, since consistency among raters was 
not anticipated, it was decided that the reasons for using different assignments were more convincing. 

Following that meeting, Advisory Committee members contacted a number of law schools to 
determine which might be interested in participating and ultimately a total of 12 schools were identified. 
Within each institution, an instructor was designated to obtain student consent for participation in a study, 
to collect writing samples, to rate the samples with respect to overall quality, and provide detailed 
commentaries on the samples. Although students could be identified by their instructors, they would 
remain anonymous to the Advisory Committee and to those at Educational Testing Service. Additionally, 
the ratings of writing samples conducted by instructors would be kept confidential; only Educational 
Testing Service, where ratings would be tallied, would have information linking specific participating 
instructors with specific ratings. Summary analyses of the ratings would be conducted, however. 

Collection of Writing Samples 

Data collection began in the fall of 1991 with the collection of 237 student legal memoranda from 
12 different law schools. These law schools were diverse with respect to geographical location, type, and 
student selectivity. Instructors and professors rated the samples with respect to overall quality and 
provided detailed commentaries in the form of annotations and written and tape-recorded commentaries. 

With the exception of one institution where the student samples used were written in the spring 
of 1991, all legal writing samples were written in the fall 1991 semester of the first year of law school. 
The samples were all typewritten and varied in length (double-spaced) between 5 and 15 pages. The 
samples were responses to a variety of assignments, each unique to the institution from which it came, 
but they were similar in that they were all legal memoranda written in response to similar assignments. 
Participating professors obtained student consent for use of their writing samples by means of the consent 
statement given in Appendix A. In addition to annotated copies of the memoranda, unmarked copies were 
made for use in obtaining ratings from other legal writing instructors and professors. 

Taxonomy Development 

The writing samples and commentaries were sent to Educational Testing Service where humanities 
specialists and legal consultants developed an initial draft taxonomy of elements of the legal memorandum. 
The taxonomy development began by engaging two legal consultants in the task of reading the 237 
samples collected, listening to tape recordings of instructor comments, reading instructor annotations and 
written comments, and categorizing the comments made. One of the legal consultants was a retired lawyer 
with membership in both the New York and New Jersey bars; the other was a recent law school graduate. 

Once an initial draft taxonomy was formulated, the samples and commentaries were next analyzed 
by humanities specialists at Educational Testing Service. The humanities specialists concentrated on those 
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aspects of the taxonomy concerned with English composition. A revision of the draft taxonomy was then 
made and distributed to members of the Advisory Committee on Legal Writing for comment. Comments 
from the Advisory Committee resulted in two revisions of the taxonomy, the final version of which is 
included in this report as Appendix B. 

Ratings of the Samples 

An initial global rating on a 1 (low) to 6 (high) scale was obtained from the instructors who 
collected the writing samples. A second global rating, as well as a first set of taxonomy element ratings, 
was obtained by sending clean, unmarked, copies of the samples to instructors participating in the study 
in institutions other than those where the samples were collected along with a taxonomy element rating 
form. A third set of global ratings was obtained from the legal consultants, although they did not rate the 
taxonomy elements. A fourth set of global ratings and a second set of taxonomy ratings was obtained by 
sending clean, unmarked copies of the samples to a third legal writing instructor participating in the study 
who had not previously seen the samples along with taxonomy element rating forms. The rating forms 
used for these ratings are included in this report as Appendix C, along with the instructions for their use. 
Minor changes in the rating forms were made between the first and second taxonomy ratings. 

Scores for taxonomy elements were developed by coding the responses as follows: 

Extremely strong element (++) = 5 
Strong element (+) = 4 
Blank = 3 
Weak element (-) = 2 
Extremely weak element (— ) = 1 

The two sets of taxonomy element ratings provided by legal writing professors were then summed to 
produce taxonomy element scores. These scores could thus range from a high of 10 to a low of 2. 

Analyses Conducted 

A number of analyses of the ratings obtained were conducted to examine frequency distributions 
of the ratings, reliability of the ratings, what elements of the taxonomy were most frequently viewed as 
positive or negative, relationships among elements of the taxonomy, relationships between elements of the 
taxonomy and global ratings of the samples, and the underlying structure of the elements of the legal 
memorandum. 

Correlational statistics were computed to examine relationships among variables and regression 
analyses to determine what elements contributed most significantly to the global ratings. A series of factor 
analyses were conducted. Exploratory factor analyses including principal component analyses and oblique 
rotations (specifically OBLIMIN) assuming different numbers of factors were conducted to identify initial 
factors solutions. See Harman (1967, pp. 324-334) for details of OBLIMIN solutions. The initial factor 
solutions were then followed by confirmatory factor analyses to arrive at a best fitting model using 
programs developed by Bentler (1989). 

Computer analyses of the sampled memoranda were conducted using Writer’s Workbench 
software. These analyses produced 73 variables for each memorandum including number of words 
written, readability indexes, average sentence length, average word length, percentage of simple sentences, 
percentage of complex sentences, percentage of compound sentences, percentage of compound-complex 
sentences, percentage of "to be" verbs, percentage of passive verbs, and percentage of nominalizations. 
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These Workbench variables were correlated with the global quality rating of the memoranda made by legal 
writing instructors. The means and standard deviations of Workbench variables obtained for the legal 
memoranda were also compared with those obtained for other kinds of English composition, including 
expository essays and periodical text. 

Finally, sex differences were analyzed for global quality ratings, taxonomy element ratings, and 
Workbench variables. Analyses of ethnic differences were not possible because of the small sample size. 

RESULTS OF DATA ANALYSES 

As previously described, global ratings were made of the sampled legal memoranda on four 
different occasions. The first global rating was made by the legal writing instructors who provided the 
samples, the second by legal writing instructors in institutions different from those where the samples 
originated, the third by legal consultants engaged by Educational Testing Service, and the fourth by 
instructors of legal writing in institutions different from those where the samples originated and who had 
not seen the samples rated previously. Table 1 shows frequency distributions obtained from the four sets 
of ratings, labelled Score 1, Score 2, Score 3, and Score 4. Of special interest in Table 1 is the degree 
to which raters used the entire range of the score scale. Score 1, provided by the originating professors, 
had a greater percentage of "1" and "6" ratings than the other three scores, suggesting a greater confidence 
in rating at the extremes. Score 2, by contrast, had the lowest percentage of scores at the extremes and, 
consequently, more scores bunched in the mid-range. 

Score 3, provided by the legal consultants, is similar to Score 4, both with an intermediate 
percentage of scores at the extremes and in the mid-range. Note that Score 3 was assigned to only 236 
samples, while the other scores were assigned to all 237 samples. The reason for the difference in 
numbers rated is that one of the samples was of poor legibility and for that reason the legal consultants 
chose not to provide a rating for it. 
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Table 1. Frequency Distributions of Global Scores 



Global Score 


Value 


Frequency 


Percent 


Score 1 


1 


32 


13.5 




2 


46 


19.4 




3 


40 


16.9 




4 


47 


19.8 




5 


42 


17.7 




6 


30 


12.7 


Total 




237 


100.0 


Score 2 


1 


18 


7.6 




2 


46 


19.4 




3 


63 


26.6 




4 


63 


26.6 




5 


32 


13.5 




6 


15 


6.3 


Total 




237 


100.0 


Score 3 


1 


23 


9.7 




2 


36 


15.3 




3 


61 


25.8 




4 


48 


20.3 




5 


45 


19.1 




6 


23 


9.7 


Total 




236 


100.0 


Score 4 


1 


21 


8.9 




2 


49 


20.7 




3 


50 


21.1 




4 


55 


23.2 




5 


40 


16.9 




6 


22 


9.3 


Total 




237 


100.0 



Note: 

Score 1 was assigned by the professor or instructor who provided the sample. 

Score 2 was assigned by a professor or instructor of legal writing in an institution 
other than that where the sample originated. 

Score 3 was assigned by a legal consultant. 

Score 4 was assigned by a professor or instructor of legal writing in an institution 
other than that where the sample originated but who had not previously seen 
the sample. 
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Table 2 shows the frequency distribution of global scores when all four scores were summed. 
Since each score had a range from 1 to 6, the range of the summed score is 4 to 24. 



Table 2. Frequency Distribution of Global Score Sum 



Value 


Frequency 


Percent 


4 


3 


1.3 


5 


1 


.4 


6 


7 


3.0 


7 


8 


3.4 


8 


12 


5.1 


9 


10 


4.2 


10 


16 


6.8 


11 


17 


7.2 


12 


21 


8.9 


13 


17 


7.2 


14 


19 


8.1 


15 


16 


6.8 


16 


21 


8.9 


17 


19 


8.1 


18 


9 


3.8 


19 


19 


8.1 


20 


9 


3.8 


21 


5 


2.1 


22 


4 


1.7 


23 


2 


.8 


24 


r . 


.4 


Total 


236 


100.0 



‘The single memorandum receiving the maximum score from all four 
readers is included in this report as an exemplar in Appendix D, along 
with the assignment to which it responds and instructor commentary. 

Rater Consistency 

Table 3 presents an analysis of global score discrepancies for different rating pairs. The number 
and percentage of global score discrepancies at different levels of absolute values are shown. For 
example, in the first row of figures it can be seen that four ratings (2 percent) differed by as much as five 
points (on the 1 to 6 scale) between the first global rating and the fourth global rating. In the second row 
of figures in Table 3, it is seen that the first and fourth ratings differed 1 1 times out of 237 pairs (5 
percent) by four points on the 1 to 6 scale. The last row of figures in Table 3 shows the number of times 
that perfect agreement occurred between ratings. The second and third global ratings (DIFF23), for 
example, agreed perfectly for 74 memoranda, or 31 percent. 
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Table 3. Global Score Discrepancies: Absolute Values of Differences for Global Ratings 



Number and Percentage for Score Pairs 



Score 

Difference’ 


DIFF12 
N = 237 


DIFF13 
N = 236 


DIFF14 
N = 237 


DIFF23 
N = 236 


DIFF24 
N = 237 


DIFF34 
N = 236 


5 


0 


1 


4 (2%) 


0 


0 


1 


4 


4 (2%) 


6 (3%) 


1 1 (5%) 


5 (2%) 


2 (1%) 


5 (2%) 


3.5 


0 


0 


0 


1 


0 


0 


3 


19 (8%) 


21 (9%) 


24 (10%) 


23 (10%) 


30 (13%) 


18 (8%) 


2.5 


1 


1 


1 


1 


0 


0 


2 


66 (28%) 


58 (25%) 


46 (19%) 


45 (19%) 


39 (16%) 


62 (26%) 


1.5 


2 (1%) 


0 


0 


0 


3 (1%) 


0 


1 


90 (38%) 


102 (43%) 


95 (40%) 


86 (36%) 


98 (41%) 


95 (40%) 


0.5 


1 


0 


1 


1 


1 


1 


0 


54 (23%) 


47 (20%) 


55 (23%) 


74 (31%) 


64 (27%) 


54 (23%) 



‘Score differences with fractions resulted because some raters made fractional ratings. 
Abbreviations: 

DIFF12 = First global score vs. second global score 

DIFF13 = First global score vs. third global score (legal consultants) 

DIFF14 = First global score vs. fourth global score 

DIFF23 = Second global score vs. third global score (legal consultants) 

DIFF24 = Second global score vs. fourth global score 

DIFF34 = Third global score (legal consultants) vs. fourth global score 



These discrepancies and agreements indicate no systematic differences between the legal writing 
professors and the legal consultants. In fact, the greatest differences occurred between the legal writing 
professors who provided the first global rating and those who provided the fourth. Table 3 shows that 
a total of 174 rating pairs out of a total of 1419 differed by three points or more, or 12 percent. Among 
the professors of legal writing, there were 94 differences of three or more scale points out of 71 1 score 
pairs, or 13 percent. Score pairs involving a legal writing professor and a legal consultant had 81 
differences of three or more out of a total of 708 score pairs, or 1 1 percent. 

These rating inconsistencies are similar to those observed by French (1962) of English composition 
class essays on which the present investigation was modelled. As noted earlier, in that study, 300 
expository essays were rated on a nine-point scale and 101 of the essays received all nine possible scores; 
no paper received less than five different grades. Inconsistencies were expected in the model study, as 
well as in the present study, because of previous research of this type. Slightly more inconsistency than 
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has been usually observed would be expected, however, because in both studies readers were not convened 
in a conference setting so that rating standards could be discussed and agreed upon. For the present study, 
additional factors that contributed to inconsistencies include: 

1. Only those instructors who provided the legal memoranda samples were intimately 
familiar with the law involved in the memoranda. Other instructors were provided with 
the same materials as the students who wrote the memoranda and achieved further 
information from the 20 or so memoranda that they read, but instructors who provided 
the samples had in some cases used the same assignment for several years. 

2. There was inconsistency in the weighting attached to specific elements of the memoranda. 
For example, some instructors weighted the legal analysis heavily while others perhaps 
expected less legal skill from first-semester law students and thus weighted traditional 
English composition considerations more heavily. 

Table 4 gives comparisons of score discrepancies for individual readers. Recall that there were 
12 legal writing instructors and two legal consultants who read and scored the samples. In Table 4, the 
readers have been given code numbers from 1 to 14. All comparisons in the table are based on scores 
assigned to the same set of legal memoranda. Most readers read three sets of 20 legal memoranda, each 
set from a different institution, although two of the readers read two sets one reader read four sets. The 
Average Difference, the second column in Table 4, was determined by computing each reader’s average 
for a set, computing the difference between each reader’s average for a set and the overall reader average 
for that set, and then averaging the differences obtained over all sets scored by a given reader. The 
second column in Table 4, Percent Discrepant, represents the percentage of times a reader’s score differed 
from another reader’s score by three or more points. Thus a "discrepant score" is viewed as one with 
three or more points difference with another reader of the same sample. The final column of Table 4, 
Percent High, was determined by computing the proportion of discrepant scores where an individual reader 
was "high" in comparison with the other reader. The "Average r" is the average correlation between a 
reader and other readers who read the same memoranda. 

The individual readers are ranked in Table 4 with respect to their estimated leniency, or tendency 
to score high. Reader #1, with an Average Difference of +.42 was clearly the most lenient by these 
estimates. But while Reader #l’s Percent Discrepant of 12 percent was not the highest, this reader was 
"high" in all (100 percent) large discrepancies with other readers. At the other extreme in Table 4, Reader 
#14 was least lenient (or most strict) with an Average Difference of -.40 and only 32 percent of discrepant 
scores higher than the paired reader (or 68 percent lower discrepant scores). 

The average correlations in Table 4 indicate the level of agreement with other readers, but these 
correlations are influenced to some degree by the particular memoranda that different readers rated. 
Reader #12 had the highest average correlation (.56) and readers #1 and #11 tied for second place (r = 
.54). Reader #14 also agreed relatively well with other readers (r = .51). Readers #8 and #9 had the 
lowest average agreement with other readers. 
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Table 4. Global Score Comparisons for Individual Readers 



Reader # 


Average 

Difference 


Percent 

Discrepant 


Percent 

High 


Average 

r 


1 


+.42 


12 


100 


.54 


2 


+.34 


12 


80 


.30 


3 


+.21 


10 


50 


.34 


4 


+.14 


10 


68 


.37 


5 


+.04 


9 


53 


.45 


6 


+.04 


10 


58 


.40 


7 


-.01 


10 


41 


.39 


8 


-.05 


15 


41 


.23 


9 


-.09 


15 


52 


.26 


10 


-.13 


12 


41 


.33 


11 


-.15 


12 


41 


.54 


12 


-.26 


8 


10 


.56 


13 


-.29 


16 


29 


.32 


14 


-.40 


12 


32 


.51 



Notes: 

1. The "Average Difference" was determined by computing each reader’s average for a set of 
legal memoranda, computing the difference between each reader’s average for a set and the 
overall reader average for that set, and then averaging the differences obtained over all sets 
scored by a given reader. Most readers scored three different sets of 20 memoranda. 

2. "Percent Discrepant" is the proportion of paired comparisons where a reader’s score was 
three or more points (on the six-point scale) different from another reader who read the 
same memorandum. 

3. The "Percent High" is the proportion of paired comparisons where a reader’s discrepant score 
was higher than the paired reader’s score. 

4. The "Average r" is the average correlation between global scores for readers reading the same 
memoranda. 



Table 5 provides comparative information by institution as well as an indication of what emphases 
specific readers had in their ratings. Emphasis was determined by computing correlations between global 
scores and taxonomy element ratings within institution. Table 5 shows that the highest correlation 
between any two readers who provided taxonomy element ratings was between readers #5 and #9 in their 
evaluations of the memorandum from Institution #5 (r = .84). These two readers both gave heavy 
emphasis to "Research," "Analysis," "Organization," and "Style." Neither gave any emphasis to 
"Conclusion" or "Assignment." They were slightly different in their emphasis on "Brief Answer" and 
"Mechanics." The only large difference in emphasis for readers #5 and #9 was on "Questions Presented," 
which reader #5 gave heavy emphasis and reader #9 no emphasis. Readers #10 and #12 also agreed 
substantially in their evaluation of the memorandum from Institution # 1 (r = .75). They both gave either 
some or heavy emphasis to "Research," "Analysis," "Organization," "Style," and "Mechanics." 
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Table 5. Reader Comparisons of Taxonomy Category Emphasis 



Taxonomy Category 












Institution 














1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


Avg. reader r .53 


.24 


.36 


.18 


.72 


.58 .60 


.10 


.17 


.34 


.33 


.52 


Score 2 X Score 4 r .75 


.42 


.56 


.00 


.84 


.50 .43 


.29 


.27 


.00 


.35 


.52 


Questions Presented 


10 


14 


8 




1 


11 


1 


7 


II 


5 


1 


11 














14 




13 




8 






Brief Answer 


10 


14 


6 


8 


1 


II 




7 


9 




i 












II 


9 


14 




13 






I 




Statement of Facts 


10 


1! 




8 


1 


14 


1 


7 




5 




11 










13 


9 




3 


13 




8 




1 


Research 


10 


2 


i 


0 


i 


11 


3 


I 


1! 


! 


3 


1 




12 


m 


8 


13 


1 


n 




m 




8 


6 


11 


Analysis 


m 


2 


6 


8 


1 


m 




i 


9 


5 


3 


1 




§§ 


li 


8 


13 




§1 


1 


ii 




8 


6 


!! 


Conclusion 


12 




6 


8 




ii 


1 


11 


9 


5 


1 


1 








8 








3 






8 






Organization 


10 


2 


6 


8 


1 


n 


1 


7 


n 


8 


1 


l 




12 


14 


I 


13 


1 


14 


3 


11 


ll 




1 


n 


Style 


10 


2 


6 


8 


5 


|| 


1 


7 


9 


5 


1 


l 




12 


14 


1 


13 


1 


II 


3 


11 


10 


1 


6 


n 


Mechanics 


10 


14 


6 


1 


1 


11 


3 


7 




1 


3 


|1 




12 




| 




9 


14 




ii 






1 




Assignment 


10 




6 


8 






| 


7 











Notes: 



1. The "Average reader r" is the overall average correlation between all global scores for an 
institution. 

2. The "Score 2 X Score 4 r" is the correlation of Score 2 and Score 4, or the correlation of 
global scores for readers who rated taxonomy elements. 

3. The figures inside the table are reader numbers. The appearance of a reader number indicates 
some emphasis on a taxonomy category and a shaded number indicates heavy emphasis. 



Table 5 also shows that readers #8 and #13, in evaluating the Institution #5 memorandum, and 
readers #5 and #8 , in evaluating the Institution #10 memorandum, did not agree at all (r = .00). In both 
cases, reader #8 gave a heavy emphasis to "Mechanics," while the other reader did not. Additionally, for 
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Institution #4, reader #8 gave some emphasis to "Conclusion" and "Assignment," while the other reader 
did not. Reader #8 also gave heavy emphasis to "Style" in both cases, while the other reader only gave 
moderate emphasis to "Style." Reader #8 appears to consistently give heavy emphasis to both "Style" and 
"Mechanics," as suggested also by the Institution #3 evaluations. It is thus seen that one source of 
difference among readers is a relative emphasis on style and mechanics and less emphasis on research and 
analysis. These kinds of differences may be associated with teaching patterns; instructors who teach only 
legal writing courses may differ systematically from instructors who teach primarily substantive legal 
courses but who also teach legal writing occasionally. 

Another source of rating differences suggested by Table 5 is an interaction between reader and 
institution (or memorandum assignment). For example, the overall average correlation for all readers 
associated with Institution #8 is only .10. Close examination of the global ratings made for Institution 
#8 shows that the ratings made by the instructor who collected the samples, and who thus knew the 
assignment best, correlated negatively with all other ratings. In other words, there was strong 
disagreement between the evaluations of quality made by the instructor who knew the assignment best and 
others who rated it. Table 5 indicates that readers #7 and #13 both gave heavy emphasis to "Research," 
"Analysis," and "Conclusion," and some emphasis to all other categories with the exception of 
"Assignment." Nevertheless, their correlation of only .29 indicates considerable disagreement on global 
scores assigned. It may be that there was something about the assignment in Institution #8 that confused 
readers. 

Frequency Analyses of Taxonomy Elements 

Of the 53 taxonomy elements identified for the legal memorandum, which elements are most often 
rated either positively or negatively by legal writing professors? Table 6 summarizes the results of two 
different ratings of the 237 legal writing samples by ranking the importance of the elements with respect 
to the average frequency with which they were rated either positively or negatively by instructors of legal 
writing. The element most commonly rated either positively or negatively was "Issue and key fact 
identification." Of the 474 rating forms completed (237 x 2), 81.4 percent contained a mark indicating 
that this element was perceived positively or negatively in a given sample. The second element most 
commonly rated either positively or negatively was "Citations (use and form)," with 69.0 percent of forms 
completed indicating the importance of citations. Four elements followed these first two in frequency: 
"Legal expression and context," "Relevance of facts stated," "Application of law to facts," and "Transition, 
flow, and logical continuity." Rounding out the top ten in frequency were: "Authority description and 
understanding, "Analogy and comparison of facts," "Relationship to questions presented," and "Clarity." 

Since the instructions for the rating forms discouraged rating major categories of the taxonomy 
such as "Discussion" and "Writing," it is not surprising that these major categories were not cited 
frequently as positive or negative aspects of samples. Subsequent analyses will show, however, that when 
the major category ratings are combined with the ratings of subelements of those categories, these major 
categories are indeed important. 
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Table 6. Elements of the Taxonomy Rated Positively 
Negatively Most Frequently 


or 


Taxonomy Element 


Percentage 


Rank 


Issue and key fact identification 


81.4 


1 


Citations (use and form) 


69.0 


2 


Adequacy of detail 


64.0 


3 


Legal expression and context 


63.7 


4 


Relevance of facts stated 


63.3 


5 


Application of law to facts 


63.0 


6 


Transition, flow, and logical continuity 


61.8 


7 


Authority description and understanding 


58.6 


8 


Analogy and comparison of facts 


58.4 


9 


Relationship to questions presented 


58.4 


9 


Clarity 


57.3 


10 



Another way of looking at the frequency of taxonomy element ratings is to compare a set of 
highly-rated memoranda with a set of poorly-rated memoranda. Additionally, if positive and negative 
ratings are considered separately, a picture of the perceptions of raters emerges. Table 7 presents a 
comparison of 50 of the best memoranda (those with global composite scores of 18 and above) with 58 
of the worst (those with global composite scores of 10 or below). 

Table 7 shows that, for the 50 best memoranda, "Application of law to facts" and "Clarity" 
received the highest positive ratings. "Transition, flow, and logical continuity" was a close third with 44 
net score points. Some of the elements have a high percentage of positive ratings, however, because raters 
had a tendency to rate them positively in many memoranda. "Relevance of facts stated," "Accuracy of 
facts," "Issue identification," and "Adequacy of detail," for example, received more positive than negative 
ratings even in the set of 58 worst memoranda. Consequently, positive ratings cannot necessarily be 
interpreted as meaning that an element was an important influence on the overall quality rating for a 
memorandum. 

Taxonomy elements receiving the most negative ratings in the 58 worst memoranda were 
"Completeness of explanation," "Analogy and comparison of facts," and "Application of law to facts." 
The worst memoranda also tended to have more negative ratings for mechanics, with "Grammar and 
usage" and "Citations" often being viewed negatively. As for the positive ratings, however, there was also 
a tendency to rate some elements negatively in most memoranda. "Citations," for example, received more 
negative than positive ratings even in the 50 best memoranda thus indicating that citations did not 
significantly affect global scores for these memoranda. 
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Table 7. Comparison of 50 Best and 58 Worst Memoranda 



Taxonomy Element 


Percent 

Positive 

Ratings 


Negative 

Ratings 


Net 

Score 


50 Best Memoranda 


Application of law to facts 


+46 


0 


+46 


Clarity 


+48 


-2 


+46 


Transition, flow, etc. 


+46 


-2 


+44 


Relevance of facts stated 


+44 


-2 


+42 


Accuracy of facts 


+40 


0 


+40 


Use of key facts 


+38 


0 


+38 


Authority selection 


+40 


-2 


+38 


Introductions and thesis statements 


+42 


-4 


+38 


Issue identification 


+44 


-8 


+36 


Adequacy of detail 


+36 


-2 


+34 


58 Worst Memoranda 


Completeness of explanation 


+3 


-52 


-49 


Analogy and comparison of facts 


0 


-48 


-48 


Application of law to facts 


0 


-46 


-46 


Transition, flow, etc. 


+2 


-43 


-41 


Grammar and usage 


+2 


-41 


-39 


Citations 


+5 


-41 


-36 


Support for statements 


+2 


-38 


-36 


Authority description 


+2 


-38 


-36 


Clarity 


+2 


-34 


-32 


Treatment of counterarguments 


+5 


-36 


-31 


Application of rationale 


+2 


-33 


-31 



Notes: 

1. Percent positive ratings was computed as the percentage of memoranda in the 
group (Best or Worst) with an element rating of 8 or more. 

2. Percent negative ratings was computed as the percentage of memoranda in the 
group (Best or Worst) with an element rating of 4 or less. 
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Correlations of Global Scores and Taxonomy Element Scores 

Of the elements of the taxonomy, which correlate most with the global quality ratings of the 
samples? In other words, ignoring the frequency with which elements were rated either positively or 
negatively, what is the relationship between element scores and global quality ratings. This would seem 
to be a more important consideration than the simple frequency with which elements were cited, and it 
avoids the interpretational problems encountered in the above analyses of positive and negative ratings. 
Two different types of correlations were computed: The first correlation computed was that between the 
taxonomy element scores and the global ratings made by two legal writing instructors who provided the 
taxonomy element ratings. This first correlation would be expected to yield the highest figures because 
all the information involved came from the rating forms. The second correlation computed was that 
between the taxonomy element scores (provided by two legal writing instructors) and the sum of all global 
scores (including those made by the instructors who initially provided the samples and those made by the 
legal consultants). These two correlations were then averaged to arrive at a ranking of the most important 
correlates of global scores. 

Table 8 shows those elements of the taxonomy that correlated highest with the global ratings. The 
two correlations given, r, and r 2 , differ because of the global ratings used for each. The first correlation, 
r„ is between the taxonomy element scores and the sum of the two global ratings made at the same time 
as the taxonomy elements were rated (using the rating form). The r, correlations tend to be higher 
because all of the scores being correlated came from the rating form. The second correlation, r 2 , involves 
scores obtained from the rating form as well as scores obtained before the rating form was developed. 
That is, it involves the global scores assigned by the legal writing professor who collected the sample 
memoranda and the global scores assigned by the legal consultants. The stronger relationship indicated 
by the first correlation may have occurred because raters possibly assigned taxonomy element scores that 
tended to support their global scores. 



Table 8. Elements of the Taxonomy That Correlated Highest 
With the Global Quality Ratings 



Taxonomy Element 


h 


h 


Clarity 


.66 


.61 


Application of law to facts 


.66 


.58 


Transition, flow, and logical continuity 


.66 


.56 


Use of key facts 


.58 


.56 


Authority description and understanding 


.59 


.51 


Analogy and comparison of facts 


.55 


.53 


Completeness of explanation 


.57 


.46 


Organization 


.54 


.49 


Introductions and thesis statements 


.55 


.47 


Authority selection and relevance 


.51 


.47 



Symbols and abbreviations: 

r, = Correlation between the sum of global scores #2 and #4 and the sum of taxonomy 
element scores for the same readers 

r 2 = Correlation between the sum of all four global scores and the sum of taxonomy 
element scores 
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From the perspective of its relationship with the global quality ratings, "Clarity" tended to correlate 
best with the global ratings followed closely by "Application of law to facts" and "Transition, flow, and 
logical continuity." This view of importance is veiy different from that indicated by the frequency 
analyses, where these same three elements ranked sixth, seventh, and tenth, respectively, in Table 6. The 
elements ranked highest with respect to frequency do not appear at all in the top ten correlates with global 
ratings shown in Table 8. Obviously, some elements are often viewed positively or negatively by legal 
writing professors even though they do not correlate highly with global ratings. "Citations" represents a 
good example of this phenomenon. Although 69 percent of the samples were rated either positively or 
negatively on citations, the use and form of citations correlated only .32 with the global ratings and thus 
would not appear (from the correlations) to be of major importance in determining the global scores for 
the samples, on average. 

Regression Analyses 

It was noted earlier that the instructions which accompanied the rating forms discouraged the 
rating of major categories of the taxonomy. While this procedure was essential for determining the 
importance of subelements of the taxonomy, the relative importance of major categories of the taxonomy 
is of interest. Cumulative scores within major categories can be analyzed through multiple regression 
analysis. Table 9 presents the results of multiple regression analyses when elements of the major 
categories of the taxonomy are combined. The analyses of Table 9 would indicate that "Discussion" and 
"Writing" are the most important of the major categories on the rating form. The most important part of 
"Discussion" is "Analysis," and the most important part of "Writing" is "Organization" (which includes 
also introductions, flow, paragraphing, and use of headings). The analysis of Table 9 suggest that 
"Questions Presented," "Brief Answer," "Statement of Facts," and "Conclusion" are less important 
taxonomy categories. 

Table 9. Multiple Correlations of Global Writing Sample Scores and Taxonomy Categories 



Taxonomy Category 


R , 


r 2 


Questions Presented 


.40 


.39 


Brief Answer 


.39 


.33 


Statement of Facts 


.34 


.34 


Discussion 


.78 


.74 


Research 


.66 


.65 


Analysis 


.74 


.70 


Conclusion 


.50 


.48 


Writing 


.77 


.73 


Organization 


.71 


.66 


Style 


.68 


.66 


Mechanics 


.52 


.52 




Notes: 

R, = Multiple correlation of taxonomy category score and the sum of 
global scores #2 and #4 

R 2 = Multiple correlation of taxonomy category score and all four global 
ratings 
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Similar to what was observed in Table 8, the R, correlations in Table 9 tend to be higher than the 
R 2 correlations because the ratings involved all came from the rating form. 

Another approach to the multiple regression analysis is to predict the global quality ratings using 
all taxonomy elements, whether they are major categories or subelements. Many of the elements will be 
found not to contribute statistically to the prediction because of their similarity (statistically) to other 
elements. The general approach is to put all of the available predictor variables in the regression, and then 
to remove those that do not make a unique contribution. Table 10 gives the results of such an analysis 
in which the sum of global scores 2 and 4 is predicted from the sum of the taxonomy element scores 
provided by the same readers. Only eleven of the elements are required to maximize the statistical 
prediction of the global quality ratings. This maximization is represented in the multiple correlation 
coefficient of .85. Further addition of elements in the regression does not increase the multiple correlation 
beyond this figure, and thus other elements are not required to maximize the level of prediction possible. 

Table 10. Prediction of the Sum of Score 2 and Score 4 Using 
Taxonomy Element Scores Provided by the Same Readers 



Taxonomy Element 


Multiple 

Correlation 


Regression 

Weight 


P' 


Issue and key fact identification 


.85 


.09 


.02 


Statement of facts 




.08 


.03 


Authority description 




.10 


.01 


Application of law to facts 




.19 


<.01 


Focus 




.09 


.03 


Conclusion 




.15 


<.01 


Organization 




.17 


<.01 


Transition, flow, and continuity 




.16 


<.01 


Clarity 




.16 


<.01 


Grammar 




.09 


.04 


Punctuation 




.11 


.01 



'The probability that an element does not make a statistically 
significant unique contribution in the prediction. 



Of the eleven elements that contribute to the prediction, the greatest weight in the prediction 
occurred for "Application of law to facts." "Organization," was second in weight, and "Flow" and 
"Clarity" tied for third place in terms of weight, even though "Conclusion" was a close fourth. This 
analysis suggests that, rather than rating all of the taxonomy elements, it would be possible to rate only 
eleven of them and still obtain a good prediction of the global quality ratings. These results have 
implications for rating forms used in evaluating student legal memoranda. Similar results were obtained 
when the sum of all four global quality ratings were predicted. 
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The variables included in Table 10 were those making a statistically significant contribution in 
the prediction of global scores. Statistical significance does not necessarily mean practical significance, 
however. Table 1 1 shows that a reasonably good prediction can be made using only seven taxonomy 
elements. Removing four variables from the analysis of Table 10 reduces the multiple correlation by only 
.01 , to .84. Thus, it would appear that some of the elements making significant contributions, statistically, 
in Table 10 do not make a practically significant contribution. 

It is of interest to compare Table 1 1 with Table 8, where the top 10 correlates with the global 
ratings are listed. Two of the variables making unique contributions in Table 11, "Conclusion" and 
"Punctuation," do not appear among the top 10 correlates in Table 8. The reason for this apparent 
inconsistency is that "Conclusion" and "Punctuation" contribute something unique to the prediction of the 
global ratings that is not represented in the top ten correlates. Since some of the top ten correlates are 
themselves highly correlated, the top ten correlates would not predict as well as the seven elements of 
Table 1 1 because some of the variance would be redundant. 

Table 11. Prediction of the Sum of Score 2 and Score 4 Using Only Seven Taxonomy Elements 



Taxonomy Element 


Multiple 

Regression 


Regression 

Weight 


P' 


Authority description 


.84 


.15 


<.01 


Application of law to facts 




.26 


<.01 


Conclusion 




.16 


<.01 


Organization 




.19 


<.01 


Transition, flow, and continuity 




.18 


<.01 


Clarity 




.19 


<.01 


Punctuation 




.15 


<.01 



'The probability that an element does not make a statistically significant 
unique contribution in the prediction. 



Factor Analyses 

Unlike multiple regression analysis, where the objective is to exclude variables that do not 
contribute in the prediction of another variable (such as the global quality rating), the objective of factor 
analysis is to simplify the description of data. Rather than reducing the number of variables by excluding 
those that do not contribute in the prediction of another variable, the number of variables is reduced by 
combining variables that represent some underlying dimension of the data. Although factor analysis was 
developed initially as a means of identifying psychological traits, it has been improved over the years and 
applied to a wide variety of behavioral situations. The object is to simplify the description of behavior 
by reducing the number of categories or variables to a few common factors. 



O 
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Factor analysis begins with a correlation matrix of all variables that one wishes to explain. In the 
present study, these variables are the ratings of the elements of the taxonomy. The global quality ratings 
are excluded, since there is no interest in combining them with the taxonomy elements. The first step in 
developing common factors from the taxonomy elements was to produce a correlation matrix of 51 
taxonomy elements for which two ratings were made. The two ratings were summed to produce 51 
composite variables and all of these were then correlated. One rating, "Recommendations for further 
action," was excluded because it was rated only once. Another rating, "Choice of organizational strategy," 
was combined with "organization" to produce a single composite variable for organization. 

A number of exploratory factor analyses of different types were conducted to determine the 
optimal number of factors and the variables which belonged best with each factor. Ultimately a six-factor 
solution was decided to be the best. Table 12 presents the results for this six-factor solution. Factor 1, 
accounting for the most variance in the data, loads heaviest on " Application of law to facts," "Use of key 
facts," and "Support for statements," and "Completeness of explanation." This first factor will be called 
ANALYSIS. The second most prominent factor loads heaviest on "Clarity," "Flow," and "Introductions 
and thesis statements," but interestingly, it includes "Focus," a taxonomy element originally grouped under 
"Analysis." This second factor will be called STYLE. 

Factor 3 loads most heavily on "Grammar and usage" and "Punctuation," and "Diction," and seems 
appropriate for a factor name such as "MECHANICS." Factor 4 loads most heavily on "Issues and key fact 
identification" and seems appropriately named ISSUES. Factor 5 encompasses the elements of 
"Conclusion," loading mainly on "Reasons." It is labelled CONCLUSION. The sixth factor consists of the 
elements of "Statement of facts," and will be called FACTS. This labelling, however, should not confuse 
this factor with facts as represented in ANALYSIS, a much more important factor. 

Table 13 shows intercorrelations among the factors. Factors 1 and 2, ANALYSIS and STYLE are 
quite highly correlated (r = .77), as are Factors 2 and 3, STYLE and MECHANICS (r = .67). Factors 3 and 
5, MECHANICS and CONCLUSION are not correlated at all ( r = -.02, not significant statistically). 
Factors 5 and 6, CONCLUSION and FACTS, are not correlated either (r = .08, not statistically significant). 
The remaining factor pairs have intermediate correlations ranging from .20 to .56. 



ERiC 



31 



25 



Table 12. Factor Loadings 


Taxonomy Element 






Loadings for Six Factors 




1 


2 


3 4 5 6 


Questions Presented 






62 



Issue and key fact identification . . . 

Legal expression and context 

Brief Answer 

Conciseness 

Relationship to Questions Presented 

Statement of Facts 

Relevance of facts stated 

Accuracy 

Adequacy of detail 

Objectivity .56 
Discussion 

Issue and subissue description 
Research 

Authority selection and relevance 60 

Authority description and understanding 69 



70 

58 

59 
52 
55 



.50 

.53 

.82 

.59 



Rule identification 
Analysis 

Application of law to facts 81 

Application of rationale 65 

Use of key facts 79 

Analogy and comparison of facts 73 

Treatment of distinctions 54 

Reasoning 59 

Support for statements 76 

Completeness of explanation 75 

Focus 48 

Conclusion 

Reasons 



Responsiveness to questions presented 
Comprehensiveness and thoroughness 



Writing 

Organization 62 

Introductions and thesis statements 75 

Transition, flow, logical continuity 79 

Paragraphing 51 

Use of headings and subheadings 

Style 53 

Clarity 80 

Avoidance of verbosity 54 

Sentence control 47 

Diction (word choice, precision) 67 

Tone and attitude 57 

Paraphrasing and use of quotations 54 

Mechanics 52 

Citations (use and form) 50 

Editing and proofreading 60 

Grammar and usage 77 

Punctuation 68 

Spelling 56 



Responsiveness to Assignment Directions 



.46 

.83 

.65 

.59 
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Table 13. Factor Intercorrelations 



Factor 






Factor 








1 


2 


3 


4 


5 


6 


1 


1.00 












2 


.77 


1.00 










3 


.56 


.67 


1.00 








4 


.53 


.48 


.37 


1.00 






5 


.38 


.26 


-.02 


.20 


1.00 




6 


.30 


.40 


.39 


.34 


.08 


1.00 



The factors should correlate with global scores in a way similar to their importance, and Table 
14 demonstrates that they do. When all cases are included, ANALYSIS and STYLE correlate best (.75 
and. 77), with MECHANICS third best (.57), and ISSUES, FACTS, and CONCLUSION least. Table 14 also 
includes some special analyses of special groupings of cases. In the second column, only the 105 cases 
with the lowest global scores are included. It was hypothesized that, for the lowest-scoring cases, 
MECHANICS and STYLE would take on relatively more importance, but this hypothesis was rejected. All 
of the correlations in the second column are reduced because of the reduction in variance for global 
scores, but the correlations for MECHANICS and STYLE were reduced about as much as were those for 
the other factors. 



Table 14. Factor Correlations With Global Scores 







Correlations with 




Factor 


Score 2 + 


Score 4 


Score 2 


Score 4 




All 
Cases 
(N = 237) 


Lowest 
Scores 
(N = 105) 


Selected 
Cases 
(N = 97) 


Selected 
Cases 
(N = 100) 


ISSUES 


.46 


.36 


.40 


.53 


FACTS 


.34 


.17 


.31 


.43 


CONCLUSION 


.31 


.20 


.45 


.29 


ANALYSIS 


.75 


.56 


.81 


.74 


STYLE 


.77 


.52 


.78 


.71 


MECHANICS 


.57 


.37 


.72 


.72 



The relative importance of a factor can be increased, however, by limiting the analyses to those 
institutions and readers who emphasize that factor. The third column of Table 14 was conducted only for 
those institutions where the reader who assigned Score 2 gave special emphasis to mechanics. These 
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selected Score 2 correlations increase substantially for MECHANICS. A similar analysis is given in column 
4, for Score 4, with similar results. 

Computer Analyses 

To complement the analyses conducted of instructor ratings, analyses were also conducted of the 
legal memoranda by means of a computer program. These computer analyses were conducted using the 
Writer’s Workbench system developed at Bell Laboratories. Specifically, the "Style" program of that 
system, which generates 73 variables automatically, was used. These variables were then correlated with 
global score ratings and with selected ratings of memoranda characteristics. Comparisons of variable 
averages were also made with other samples of writing. 

Table 15 gives selected results from the Workbench analyses. More detail from these analyses 
is given in Appendix E. The readability grade level indexes represent attempts to scale reading difficulty 
so that textbooks and other materials can be evaluated with respect to their appropriateness for use in 
different grades. They are based on formulas that include word and sentence-length variables, estimates 
of word difficulties, and other variables. The different indexes use slightly different formulas, but all are 
similar (see, Flesch, 1974, 1979; Kincaid, 1973, for details of these methods). Table 15 shows that the 
three indexes presented suggest that the legal memoranda analyzed ranged in reading difficulty from about 
the eighth grade level to "grade 17." Taken together, the legal memoranda averaged between the eleventh 
grade level and somewhat above the 12th grade level. 

The overall average sentence length in the legal memoranda was about 19 words, although one 
memorandum averaged only about 1 1 words per sentence while another averaged as many as 30 words 
per sentence. The percentage of simple sentences ranged from a low of 22 percent in one memoranda to 
80 percent in another; complex sentences ranged from 13 percent to 60 percent; compound sentences 
from one percent to 1 8 percent, and compound-complex sentences from one percent to 29 percent. 

Of verb types, "to be" verbs ranged from 14 percent to 45 percent and passive verbs from five 
percent to 29 percent. Other word types ranged from 7.8 percent to 13.2 percent for prepositions, 1.3 
percent to 4 percent for conjunctions, 1.9 percent to 6.7 percent for adverbs, 24.1 to 35.5 percent for 
nouns, 10.8 to 25.6 percent for adjectives, 1.3 percent to 6.4 percent for pronouns, and one percent to 5 
percent for nominal izations. 
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Table 15. Summary of Selected Variables from Writer’s Workbench 

(N = 176) 



Variable 


Average 


Minimum 


Maximum 


Readability Grade Level Indexes 


Kincaid 


11.3 


7.9 


16.3 


Coleman - Liau 


11.2 


9.0 


13.7 


Flesch 


12.6 


8.6 


17.0 


Sentence Information 


Number of sentences 


148.0 


69.0 


259.0 


Average sentence length (words) 


19.3 


11.5 


30.0 


Percent of simple sentences 


45.4 


22.0 


80.0 


Percent of complex sentences 


36.6 


13.0 


60.0 


Percent of compound sentences 


6.0 


1.0 


18.0 


Percent of compound-complex sentences 


11.9 


1.0 


29.0 


Word Information 


Number of words 


2818.0 


1319.0 


4803.0 


Average word length (letters) 


4.9 


4.5 


5.3 


Percent of "to be" verbs 


33.6 


14.0 


47.0 


Percent of passive verbs 


15.7 


5.0 


29.0 


Percent of nominalizations 


3.0 


1.0 


5.0 



Notes: 

1 . The percent of "to be" verbs is computed by Workbench as a percent of total verbs. 

2. The percent of passive verbs is computed by Workbench as a percent of total 
non-infinitive verbs. 

3. The percent of nominal izations is computed by Workbench as a percent of total 
words. 



Table 16 shows correlations between the total of the four global ratings and selected Workbench 
variables. The only statistically significant correlations between the total global scores and the Workbench 
variables are for Number of Sentences (.14) and Number of Words (.19); it is commonly observed that 
amount written correlates with judgments of writing quality (Breland & Jones, 1982; Breland et al., 1987), 
and such correlations are substantial when time to complete the writing task is limited. Since the student 
legal memoranda were all take home assignments, some with page-length limitations, it would not be 
expected that length would correlate highly with overall quality. Examination of correlations between 
Workbench variables and global scores within institution and for specific readers suggests that, for some 
institutions and some readers significant correlations do occur, but about half of these correlations are 
negative and half positive. Consequently, when all readers and all institutions are pooled together, the 
result is no correlation overall. 
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Table 16. Correlations Between Global Rating Total Score 
and Selected Writers’ Workbench Variables 
(N = 176) 
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Variable 


r 


Readability Grade Level Indexes 


Kincaid 


.05 


Coleman - Liau 


.06 


Flesch 


.02 


Sentence Information 


Number of sentences 


.14 


Average sentence length (words) 


.07 


Percent of simple sentences 


-.02 


Percent of complex sentences 


-.01 


Percent of compound sentences 


.11 


Percent of compound-complex sentences 


-.01 


Word Information 


Number of words 


.19 


Average word length (letters) 


.05 


Percent of "to be" verbs 


-.05 


Percent of passive verbs 


-.04 


Percent of nominalizations 


-.06 



Note: See notes for Table 15 



Since "Clarity" was an important variable in the prediction of the global quality ratings, it was of 
interest to examine this rating variable in relation to the Workbench variables to determine if clarity might 
be in part explained by some of the Workbench variables. Table 17 shows correlations between the 
"Clarity" rating variable and selected Workbench variables. With the exception of the percentage of 
compound sentences (r = .18), none of the Workbench variables appear to correlate with "Clarity." It is 
thus suggested that clarity in the legal memoranda was a characteristic too subtle to be detected by a 
computer. The modest relationship with the percentage of compound sentences may occur either because 
compound sentences are more clear or because the law students who used more compound sentences just 
happened to be the same students who wrote more clearly. 
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Table 17. Correlations Between "Clarity" Ratings and 
Selected Writers’ Workbench Variables 
(N = 176) 


Variable 


r 


Readability Grade Level Indexes 




Kincaid 


.01 


Coleman - Liau 


-.01 


Flesch 


-.02 


Sentence Information 




Number of sentences 


.23 


Average sentence length (words) 


.02 


Percent of simple sentences 


-.08 


Percent of complex sentences 


-.02 


Percent of compound sentences 


.18 


Percent of compound-complex sentences 


t 

O 

o 


Word Information 




Number of words 


.24 


Average word length (letters) 


.01 


Percent of "to be" verbs 


-.09 


Percent of passive verbs 


.08 


Percent of nominalizations 


-.05 



Note: See notes for Table 15. 

How do the computer analyses of the legal memoranda compare with similar analyses of other 
types of writing? Table 1 8 presents a comparison of the legal memoranda with text sampled from several 
recent periodicals. Usually, writers for national periodicals would be considered to be more sophisticated 
writers than the average law student. Table 1 8 suggests no important differences in the average readability 
grade level of student legal memoranda and national periodicals. There are some differences in sentences. 
The average sentence length is slightly longer for the periodicals, and the average percentage of compound 
sentences is somewhat larger for the periodical text. The largest differences occur for word types, 
however. The percentage of passive verbs and the percentage of nominalizations are much larger for the 
student legal memoranda, which justifies the emphasis in legal writing textbooks on these two writing 
problems. Passive verb forms do not indicate who is doing what, and thus can be unclear ("The victim 
was assaulted"). Nominalization refers to the process of turning adjectives, adverbs, and verbs into nouns 
("Counsel made an objection to the expert’s testimony," is a nominalization of "Counsel objected to the 
expert’s testimony"). The percentage of pronouns is much less for the student legal memoranda, perhaps 
reflecting the formal nature of legal writing. 
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Table 18. Workbench Comparison of Legal Memoranda with 
48 Samples of Periodical Text 






Workbench Variable 


Legal Memos 


Periodicals 


Effect 

Size 


Mean 


S.D. 


Mean 


S.D. 


Readability Grade Level Indexes 


Kincaid 


11.3 


1.5 


11.3 


3.5 


.00 


Coleman - Liau 


11.2 


1.1 


10.9 


2.2 


.18 


Flesch 


12.6 


1.0 


11.9 


3.2 


.27 


Sentence Information 


Average sentence length (words) 


19.3 


2.9 


21.2 


5.5 


-.45 


Percent of simple sentences 


45.4 


10.2 


42.6 


12.8 


.24 


Percent of complex sentences 


36.6 


8.1 


35.6 


8.8 


.12 


Percent of compound sentences 


6.0 


2.9 


8.9 


3.9 


-.85 


Percent of compound-complex sentences 


11.9 


5.1 


12.8 


6.9 


-.15 


Word Information 


Average word length 


4.9 


.2 


4.8 


.3 


.40 


Percent "to be" verbs 


33.6 


6.6 


30.3 


5.3 


.55 


Percent passive verbs 


15.7 


5.0 


8.1 


3.5 


1.79 


Percent nouns 


28.6 


2.1 


26.5 


2.2 


.98 


Percent pronouns 


3.3 


.9 


6.5 


2.7 


-1.78 


Percent nominalizations 


3.0 


.9 


1.5 


.8 


1.76 



Rather than examining only averages, it is also of interest to compare individual legal memoranda. 
Table 19 gives a comparison of a student legal memorandum receiving the lowest global scores from all 
readers, a student legal memorandum, and a memorandum prepared by a leading law firm. The 
professional legal memorandum has higher readability grade level indexes, fewer complex sentences, more 
compound sentences, fewer "to be" verbs, more passive verbs, and more nominalizations than the student 
legal memoranda. Although it is difficult to generalize from only three samples, this comparison suggests 
that, while passive verbs and nominalizations may often be problems, they can also be used to good effect 
by sophisticated legal writers. 
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Table 19. Workbench Comparison of Three Legal Memoranda 



Variable 



Means for Three Memoranda 
Student 5-9 Student 7-20* Professional 



Readability Grade Level Indexes 



Kincaid 


10.7 


9.9 


13.0 


Coleman - Liau 


9.7 


10.1 


11.8 


Flesch 


10.3 


10.3 


14.4 


Sentence Information 


Average sentence length 


22.3 


19.1 


21.9 


Percent of simple sentences 


38.0 


35.0 


34.0 


Percent of complex sentences 


44.0 


38.0 


32.0 


Percent of compound sentences 


4.0 


11.0 


18.0 


Percent of compound-complex sentences 


14.0 


16.0 


16.0 


Word Information 


Average word length 


4.6 


4.7 


4.9 


Percent "to be" verbs 


38.0 


40.0 


32.0 


Percent passive verbs 


16.0 


17.0 


28.0 


Percent nominal izations 


2.0 


2.0 


4.0 



’This is the exemplary student legal memorandum included as Appendix D. 



Analyses of Sex Differences 

Although observed average global ratings for females were slightly higher, no statistically 
significant sex differences occurred. Some significant sex differences in the taxonomy element ratings 
were observed, however. Of the 237 legal memoranda rated, a total of 229 were identified with respect 
to the sex of the writer. The 229 memoranda for which the sex of the writer was identified included 138 
written by males and 91 written by females. Table 20 shows effect sizes (differences in standard deviation 
units) for eight taxonomy elements. All of the differences favor females, as indicated by the negative 
signs of the effect sizes computed by subtracting female means from male means and dividing by the 
average standard deviation. The largest effect size (-.33) was for a grouping of mechanics elements 
labelled "Selected Mechanics" and consisting of citation use and form, editing and proofreading, and 
grammar and usage. The next largest effect size (-.32) was for application of law to facts; however, tone 
and attitude (-.31) and accuracy of facts (-.30) also had negative effect sizes. In contrast to the taxonomy 
element ratings, there were no statistically or practically significant sex differences observed in the 
computer analyses. 

Legal writing instructors provided detailed commentaries of 237 legal writing samples written in 
the first year of law school at 12 different institutions. Comments were listed and organized into 53 
categories. From these listed and organized comments, a taxonomy of the elements of the legal 
memorandum as evidenced by the comments was constructed and from the taxonomy a rating form was 
developed. The taxonomy and the rating form were reviewed by an Advisory Committee of experienced 
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legal writing specialists and then revised. To introduce objectivity into the ratings, the rating form was 
then used to elicit judgments from legal writing professors other than those who had provided the samples. 
Judgments were made both of the overall quality of sampled legal memoranda as well as the quality and 
importance of specific elements. 



Table 20. Sex Differences in Taxonomy Elements 
(N = 229, 138 Males and 91 Females) 



Taxonomy Element 


Males 


Females 


P 


Effect 

Size 


Mean 


S.D. 


Mean 


S.D. 


Accuracy (of facts) 


6.59 


1.12 


6.90 


.97 


.03 


-.30 


Application of law to facts 


5.71 


1.52 


6.20 


1.59 


.02 


-.32 


Use of key facts 


6.09 


1.48 


6.38 


1.33 


.05 


-.18 


Tone and attitude 


5.83 


1.12 


6.19 


1.17 


.02 


-.31 


Citations (use and form) 


5.17 


1.59 


5.57 


1.49 


.05 


-.26 


Editing and proofreading 


5.30 


1.26 


5.65 


1.31 


.05 


-.27 


Grammar and usage 


5.48 


1.21 


5.80 


1.28 


.05 


-.26 


Selected mechanics 


15.95 


3.20 


17.02 


3.33 


.01 


-.33 



Notes: 



1. p is the probability that a sex difference is not statistically significant. 

2. Effect Size is the mean sex difference in standard deviations. An Effect Size of 
.20 is considered to be "small," an Effect Size of .50 is considered "medium," and 
an Effect Size of .80 "large." 

3. Selected Mechanics is the sum of Citations (use and form), Editing and 
proofreading, and Grammar and usage. 

4. A negative Effect Size indicates that the mean for females is higher than the 
mean for males. 

SUMMARY AND DISCUSSION 

As anticipated from previous studies of English composition, analyses of the judgments of the 
samples revealed some disagreement among instructors as to what constitutes good or poor legal writing. 
Nevertheless, it was possible to identify important elements of legal writing quality by considering all of 
the ratings as an aggregate. Two types of ratings were made of the writing samples collected: (1) global 
ratings of the overall quality of the samples, and (2) ratings of which elements of the taxonomy were 
considered by the raters to be either strong or weak in a particular writing sample. Raters often disagreed 
on the overall quality of a given sample as well on the elements of it that were either strong or weak. 
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Elements of the taxonomy were first examined with respect to the frequency with which they were 
rated weak or strong in samples. Second, the taxonomy elements were examined with respect to the 
relationship between ratings of them and the global quality ratings. The results showed that those 
elements commented on most frequently were not the same elements that correlated best with global 
quality ratings. Although issue identification and citation form were noted frequently as either weak or 
strong aspects of a sample, ratings of clarity, flow, and application of law to facts correlated best with 
overall quality. Ratings of issue identification and citation form were not even among the top ten 
correlates with overall quality. This observation suggests that, even though legal writing instructors 
comment frequently on certain aspects of student legal memoranda (such as citation form), these aspects 
of legal writing are not necessarily important influences on their judgments of overall quality of a sample. 

An analysis was conducted to determine which of the 53 taxonomy elements would contribute 
significantly in a prediction of overall quality, and 1 1 such elements were identified. Although eleven 
elements made statistically significant contributions in the prediction of global quality ratings, reasonable 
predictions could be made with as few as seven. The heaviest weights in the prediction occurred for 
application of law to facts, organization, flow, and clarity. The 53 elements of the taxonomy were also 
analyzed as groups in the categories used in the taxonomy. Those elements comprising the "Discussion" 
category of elements and the "Writing" category were found to be most highly correlated with overall 
quality and to be of equal importance. Less important categories, in their order of importance, were 
"Conclusion," "Questions Presented," "Brief Answer," and "Statement of Facts." 

Confirmatory factor analysis was used to examine the factor structure of the taxonomy element 
ratings. Following some initial exploratory factor analyses, a six-factor model was posited based on the 
exploratory results and the logic of the taxonomy. The six factors, in order of importance, were: 
ANALYSIS, STYLE, MECHANICS, ISSUES, CONCLUSION, and FACTS. The ANALYSIS factor, which was 
most important, was primarily associated with application of law to facts, the use of key facts, support for 
statements, and completeness of explanation. The next most important factor was STYLE, which was 
primarily associated with clarity, flow, organization, and thesis statements. The third most 
important factor, MECHANICS, represented primarily what one would expect: grammar and usage, 
punctuation, and diction. ISSUES was primarily issue identification but included also other elements of 
questions presented and brief answer. CONCLUSION and FACTS were relatively weak factors related to 
two standard parts of the legal memorandum, the conclusion and the statement of facts. 

Although it was possible to fit a factor model corresponding largely to the taxonomy element 
groupings, some deviations from the taxonomy were necessary. Rather than representing two separate 
factors, "Research" and "Analysis" were found to provide the best model fit when combined, and "Focus" 
did not fit as well with the ANALYSIS factor as it did with the STYLE factor. "Organization" did not form 
a factor of its own, as might be suggested by rating forms used by legal writing instructors, but combined 
with other elements (principally "Flow" and "Clarity") to create the STYLE factor. Finally, "Style" and 
"Mechanics," as represented in the taxonomy, did not form factors; three of the taxonomy elements 
categorized under "Style" combined with elements under "Organization" to form the STYLE factor and 
the remainder of the "Style" elements combined with "Mechanics" elements to form the MECHANICS 
factor. 



Computer analyses of the legal memoranda sampled were conducted to examine the text at a finer 
level of detail than was possible from the rater judgments. They also allowed for comparisons with other 
kinds of writing for which the same computer analyses had been conducted, and some differences were 
observed. When compared with samples of periodical text, the student legal memoranda were seen to 
have proportionally more passive verb constructions and more nominalizations. These observations 
support the emphasis given in many legal writing textbooks on passive voice and nominalizations. When 
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related to the global quality scores assigned to the student legal memoranda by legal writing instructors, 
however, little relationship was apparent between the computer-generated variables and perceived quality. 
Although there was some indication that some instructors may have been influenced in their quality 
judgments by passive verb constructions, nominalizations, and readability, most instructors were not, and 
when the judgments were aggregated, no relationships occurred. The Workbench variables were also 
correlated with "Clarity" ratings of the memoranda to determine if any of these variables would help to 
explain clarity, but no significant correlations were revealed. Thus, the Workbench analyses indicate that 
computer-generated variables are not likely to explain either global quality ratings or clarity. 

Some interesting sex differences were observed. Although no statistically significant differences 
were observed between the sexes in global quality ratings, females received significantly higher ratings 
on several of the taxonomy elements: accuracy of facts, application of law to facts, use of key facts, tone 
and attitude, citation use and form, editing and proofreading, grammar and usage, and mechanics. Why 
the global quality ratings did not also favor females is not clear. 

The analyses conducted for this study give some indication of how legal writing differs from other 
kinds of writing, but the differences are debatable. Since application of law to facts, and legal analysis 
more generally, are obviously not features of most other kinds of writing, these aspects of legal writing 
would appear represent a primary difference, even though it may be argued that similar forms of deductive 
reasoning are used in non-legal situations. Clarity, flow, and organization are terms often used in 
evaluating other types of English composition, although "transition" is a more common term than "flow." 
What appear on the surface to be traditional English composition concerns often take on new meaning in 
legal writing, however. These traditional concerns are not identical in different genres. Organizational 
requirements, for example, are different for fiction and technical writing; in legal writing, they are 
informed almost exclusively by legal analysis. Thesis statements also have a special meaning in legal 
writing. 



The analyses conducted suggest important implications for legal instruction, since some elements 
of the taxonomy seemed to be more important at low levels of performance. Although application of law 
to facts, clarity, and flow were important at both high and low levels of performance, grammar and usage, 
support for statements, and completeness of explanation appear to be especially important at low levels 
of performance. Another important implication for legal instruction resulting from these analyses is the 
types of textbooks supported. The legal writing textbooks whose content agrees most with the results of 
this study would include Calleros (1990), Neumann (1990), Shapo et al. (1991), Pratt (1990), and Teply 
(1990). Textbooks emphasizing primarily English composition topics are not supported by the results of 
this study. 

This study was based on a taxonomy of legal writing developed from law school instructors’ 
commentary on student legal memoranda. Accordingly, the taxonomy developed is representative of legal 
writing as it was taught in twelve law schools in 1991. The instructors who happened to be teaching legal 
writing in those schools at that time were not necessarily experts in legal writing, however. This fact 
raises the question of what a taxonomy of the legal memorandum would look like if it were designed 
solely by experts in legal writing. This question was addressed to the project Advisory Committee made 
up of experts in legal writing. One member of the Committee produced a draft taxonomy and, through 
review and commentary by other members of the Committee, an expert taxonomy was created. This 
expert taxonomy is presented in Table 21. 
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Table 21. Summary Taxonomy of Legal Writing Based on Expert Judgment 



Questions Presented or Issue Identification 
Identifies issues correctly 
Includes key facts and legal context 
Uses readable length 
Uses readable sentence structure 

Statement of Facts 

Provides context first, unless unnecessary 
Provides procedural posture, if any 
Organizes appropriately 

Uses only relevant facts and necessary, background data 
Provides a readable narrative 

Conclusion or Brief Answer 

Answers the Questions Presented 
Summarizes the Discussion 
Gives reasons briefly 

Is understandable to someone who does not know the analysis 
Parallels Questions Presented and organization of Discussion 

Discussion 

1 . Research 

Includes the controlling statutes and cases 

Includes other relevant primary and secondary materials 

Distinguishes controlling and persuasive authority 

2. Organization 

Uses logical large scale organization of issues and sub-issues 

Organizes each sub-issue by topics in logical order rather than case by case 

Uses introductions and thesis statements 

Provides logical transitions 

Synthesizes authority 

3. Analysis 

a. Structure 



Synthesizes a general rule or rules 
Uses rule or rules to analyze facts 

Compares facts, holdings, rules, or policies of cases to the problem 
Contrasts arguments for and against outcomes using cases as defined above 
Applies all relevant statutory language, legislative intent and statutory purpose 
Deals with relationships to common law, as necessary 



b. Strength of Analysis 

Explores the issues 

Gives sufficient reasons for conclusions 
Evaluates arguments realistically 
Substantiates all statements with authority 

4. Writing Style 
Controls sentences 

Uses concrete, active sentences 
Keeps sentences to readable lengths 
Uses proper word choice 
Limits use of quotations 
Uses coherent paragraphs 



Mechanics 



Uses correct spelling 
Uses correct grammar 
Uses correct punctuation 
Uses correct citation form 
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The expert taxonomy is similar in many ways to the empirical taxonomy developed for the present 
study, but there are some differences. Most notable in the expert taxonomy of Table 21 is that 
"Organization" and "Writing Style" are subsumed under "Discussion," and "Mechanics" is viewed as a 
major category. In the empirical taxonomy, "Organization," "Style," and "Mechanics" were subsumed 
under "Writing," a major category separate from "Discussion." In either taxonomy, all elements and 
categories are of course highly interrelated. Consequently, categories are only convenient labels for 
different aspects of the discourse. Still, the differences between the expert and empirical taxonomies are 
instructive. The experts tend to consider writing as an integral part of the legal research and analysis, not 
as a separate category. In the words of one Advisory Committee member: "Legal research is legal 
analysis is legal writing." 
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Appendix A. Student Consent Statement 



Dear Law Student: 



The Law School Admission Council is sponsoring a study of legal 
writing in which your instructor of legal writing is participating. The 
objective of the study is to observe legal writing as it is taught in U. S. law 
schools. The results of the study will be used to develop materials for 
guidance and instruction of future law students. 

A copy of one of your written assignments that you have completed 
(with your name removed) may be used in the study. Your written 
assignment will always remain anonymous, and its use in the study will not 
affect your grade in any way. 

The results of the study will benefit future law students. 

If for any reason you do not want any of your written assignments 
used for this research, please inform your instructor. 
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Appendix B. Taxonomy of Elements of the 

Legal Memorandum 



Elements of the Legal Memorandum 

(as determined from instructor commentary) 



Questions Presented 

A statement phrased in question form and properly stating the legal context of the 
issues raised in the assignment. These are concise questions which frame the major legal 
issues addressed in the memorandum and avoid minor technical questions. If there is 
more than one issue, a separate question should be used for each. 

Issue and kev fact identification . Recognition of the key facts and main issues, 
correctly identifying the main issues of the problem, and integrating the most 
legally significant facts into the framing of the questions. 

Legal expression and context . Phrasing the issue in the appropriate legal 
ter min ology and consequence (effect) and properly placing the issues and key 
facts in their legal context. 

Brief Answer 

The conclusion in response to the question, without the elaboration of facts or 
legal analysis. These are short, conclusive statements meant to answer the questions 
presented. Should parallel Questions Presented and Discussion . 

Conciseness . Directness and simplicity of expression. Answers the question by 
applying a rule of law to the issue involved. 

Relationship to Questions Presented . The degree to which the answer is 
responsive to the questions. The answer should be specific and problem-oriented; 
a broad, general statement of the law is not desired. 
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Statement of Facts 



A description of the relevant and essential details and circumstances contained in the 
statement of the assigned problem. The goal of the statement of facts is twofold: (a) to 
include enough facts to give the reader a feel for the problem, and (b) to focus on those 
facts of legal relevance. 

Relevance of Facts Stated . The selection and description of those facts which have a 
bearing or effect upon the determination of the issues presented, keeping only those 
relevant to the legal issues or necessary to the understanding of the problem. Factual 
uncertainties, the resolutions of which are legally relevant, should also be noted. 
Accuracy . Detailed and correct reporting of the facts and circumstances of the 
problem statement. There should also be no discrepancies between the statement of 
facts and the facts as given in the assignment. 

Adequacy of Detail . The extent to which the facts selected are specific and precise 
and fully illustrate the relevant activities and circumstances of the problem statement. 
Enough facts, including some not directly related to the legal issues, are included to 
(a) familiarize the reader with the circumstances of the problem and (b) establish a 
basis for comparing the facts of the problem with facts of cases used in the analysis. 
Objectivity . Reporting the facts fairly, without insinuation, suggestion, or bias. Legal 
conclusions or conclusory treatments of factual issues should be absent. The facts are 
not presented in a manner designed to favor one legal conclusion or another. 
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Discussion 



Issue and Sub-issue Description 

This description should properly and clearly state the legal issues as posed in the 
questions presented and divide the issues into their legal components and describe them. 
Research 

The examination of available legal authorities relevant to the fact situation and issues 
at hand. 

Authority Selection and Relevance . The identification of those cases which are similar 
to the issues and key facts at hand for purposes of analogy and distinction. Uses 
statutes, cases, and other legal authorities that are pertinent, controlling in their 
jurisdiction, and strongly persuasive. 

Authority Description and Understanding . Explanation of the facts and holdings of 
the cases selected and understanding of the rationale and reasoning contained in those 
cases and their relation to statutes. All authorities should be described accurately and 
in enough detail for the reader to understand why the authority is being used and to 
convey the impression that the writer understands it. 

Rule Identification and Explanation . Recognition of that authority which governs the 
deter mina tion of the issue presented. The rule of the law as expressed in the 
language of a statute or regulation or the holding of a case should be explained before 
applying it to the facts at hand. 
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Analysis 



The examination of the facts against the language, rationale, and facts of the 
governing rule/case law for the purpose of determining how the question is to be decided. 

A pplication of Law to Facts . The comparison of the facts at hand to the fact situation 
and rule/case law. This comparison should reveal similarities and distinctions upon 
which to base reasoning to a conclusion as to which rule or holding shall determine 
the issue presented. The facts of the problem are integrated into the analysis by 
relating them to the facts and policies of the ruling authorities. 

Application of Rationale . Using the reasoning and policy considerations 
supporting the holdings of the authorities in the analysis of the fact situation at 
hand. To determine the rationale of the law, one asks: "Why is the law this 
way?" Policy statements, legislative history, or explanatory dicta can all be 
applied to the facts of the problem to establish a linkage between the problem 
and the legal authority. 

Use of Kev Facts . Identification of those facts which are held to be important 
by the case authorities in their holdings so that they may be used in the 
analysis of the case at hand. All legally si gnifi cant facts from case authority or 
fact patterns in policy statements of statutes are compared to the facts of the 
problem to expand the link between the law and fact. Unlike analogy, this 
aspect merely helps to establish the factual link between the facts of the 
authorities and the facts of the problem. 
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Analogy and Comparison of Facts . Compares facts of problem with elements of 
statute or co mm on law rule; compares facts of problem with facts in judicial opinions. 
Treatment of Distinctions and Counterarguments . The recognition and handling of 
differences in fact situations which give rise to questions of the applicability of the 
rationale and rule to the fact situation at hand. Identifies lines of legal reasoning 
contrary to conclusions thoroughly discusses them. On a smaller scale, analyzes cases 
and factual differences, and shows why the rule may not apply to the problem. 
Reasoning . The judgments and choices made in the selection of the analogies, 
comparisons, and distinctions used in support of the conclusion. 

Support for Statements . The facts, authority, and rationale used to support 
statements and conclusions. Makes no statement of law or fact that is not 
backed up by a citation or explanation. 

Completeness of Explanation and Discussion . The detail and depth of the 
illustrations and assessments of fact situations and rationales used in the 
explanation of choices made in the progress toward a decision. Explains the 
analysis of each issue and subissue. 

Focus . Concentration and emphasis upon the major issues and the determining 
facts throughout the analysis keeping in mind the conclusion to be reached in 
assembling arguments. The priorities assigned to the various questions. Each 
section of the discussion should cover only one issue or subissue, without 
elements of others. 
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Conclusion 



The response to questions presented as a result of the analysis of facts and law. This 
should also illustrate the reasons supporting the response. Often used in conjunction with or 
in lieu of a brief answer, the conclusion is meant to be more inclusive than the brief answer. 
Should state the issues and show the reasoning behind the decision. 

Reasons . The description of the major factors and circumstances upon which the 
conclusion is based. The bases for the conclusion must be sound legal analysis clearly 
stated. 

Responsiveness to Questions Presented . The conclusions should be the final specific 
resolutions of the major issues. 

Comprehensiveness and Thoroughness . Explains what the answer is, but also, by way 
of a concise, on-point explanation of law, why the answer is what it is. 
Recommendations for Further Action . When appropriate, recommendations are made 
for further analysis or other action. 
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Writing 



Organization 

The mann er in which the writing is arranged so as to bring together, according to the 
issues addressed, those facts and authorities pertinent to the issues. 

Choice of Organizational Strategy . The organizational strategy should be 
appropriate for the audience and the legal problem being analyzed. 

Introductions and thesis statements . Thesis paragraphs at the beginning of each 
section and topic sentences at the beginning of each paragraph should clearly show the 
reader what is about to be discussed. 

Transition. Flow. Logical Continuity. Coherence . Between and within paragraphs and 
sections, the passage from one idea to the next should be smooth. The relationship 
between sections and subsections should be clear to the reader. Appropriate sentence 
and paragraph order aids in making transitions. 

Paragraphing . Paragraphs should be logical groupings of thoughts of reasonable 
length. Reasonable length means, at the extremes, more than one sentence but less 
than one page. 

IJse of Headings and Subheadings . As guides to the analysis, they should stand out 
from the text (boldface, etc.) and be specific and concise. They should not be used in 
place of topic sentences. Use of parallel sections when appropriate. 
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Clarity . Easily understood by the audience to which the memorandum is addressed, 
usually a senior partner. Ray & Ramsfield (1987, p. 40) say the following about clarity 
in legal writing: "Clarity means writing so the reader can follow the writing step by 
step, without wondering what a phrase means or what the point of the paper is. When 
the writing is clear, the reader can forget about the writing itself and focus instead on 
the merit of the content." 

Avoidance of verbosity . Avoids wordiness, needless detail, unnecessary information, 
needless repetitiveness, redundancy. 

Sentence Control . Strong, straightforward sentences; longer sentences structured 
effectively with sentence length varied appropriately; action verbs, parallel structure, 
and other techniques should be used when appropriate. Avoids separating subject 
from verb with intrusive phrases, unnecessary complexities in sentences, choppiness, 
run-on sentences, sentence fragments. 

Diction . Word choice that is precise, avoids jargon, achieves an appropriate level of 
formality, and avoids wordy constructions (e.g., nominalizations). Avoids vague, 
indefinite statements (e.g., "it appears that," "it would seem that," "might," "may," 
"I doubt," "suggests," "we may assume," etc.) and legalese (e.g., whereas, 
hereinafter, etc.) 

Tone and Attitude . The audience is understood and the language used is appropriate 
for that audience. Legal writing is not informal. The writing does not insult the 
reader’s intelligence. Avoids use of first person, contractions, colloqualisms, 



underlining or italics, exclamation points, rhetorical questions and sarcasm. 
Paraphrasing and Use of Quotations . Usually, quotations should be avoided through 
the use of paraphrasing. When quotations are unavoidable, they should be kept brief 
and incorporated smoothly into the text. However, the memorandum should either 
quote or rewrite, not semi-quote. 

Other . Avoids use of generic "he" or other sexist language. Avoids omission of 
articles when needed (e.g., "the" defendant). 

Mechanics 

Citations . Properly attributes all statements and uses correct Blue Book form for all 
citations. 

Editing and Proofreading . Typographical errors, missing words, gaps in the text, 
capitalization, etc., corrected. 

Grammar and Usage . Subject-verb agreement, pronoun usage, and use of proper 
tense are common problems in much writing. 

Punctuation . The rules of punctuation apply, with special attention to restrictive and 
nonrestrictive (set off by commas) phrases and clauses, use of parenthetical 
expressions, and separating the elements of a series by commas. 

Spelling . In law student papers, commonly misspelled words are "judgment" (not 
"judgement") and "argument" (not "arguement"). 

Responsiveness to Assignment Directions 

The degree to which the writer has addressed the issues presented by the instructor as 
well as the mechanical requirements such as deadlines, page and word limits, etc. 
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Appendix C. Rating Forms and Instructions 



PRINCETON. N.J. 08541 



EDUCATIONAL TESTING SERVICE 




600 - 023 -0000 

CABLE-EDUCTESTSVC 

« 

PROGRAM RESEARCH OFFICE 



November 1, 1991 



Dear [Legal Writing Professor or Instructor]: 

As we discussed on the telephone, the Law School Admission Council is 
sponsoring a study of legal writing. This study is being conducted by 
Educational Testing Service, and it is expected that about 12 law schools will 
participate. Enclosed are instructions and materials for use in the study if 
you wish to participate. 

Our Advisory Committee has agreed that instructors or professors 
participating in the study should receive payment of $500 for collecting 20 
legal writing samples, providing commentaries on these samples, keeping a 
roster of students with code numbers, and scoring 20 writing samples from 
another participating institution. An initial payment of $300 will be made 
when the 20 writing samples and instructor’s commentaries on them have been 
received (by January 10, 1992). The remaining $200 will be paid when student 
rosters and scores on 20 writing samples from another institution are received 
(by April 15, 1992). 

Enclosed are the following: 

1. Specifications for Legal Writing Samples and Instructor’s 
Commentaries . 

2. A statement addressed to law students informing them of the study 
and allowing them to object to the use of their written 
assignments if they wish to do so. Consent is implied if no 
objections are raised. Instructors will need to obtain copies of 
this statement depending upon the size of the group from which the 
20 samples are to be selected. 

3. A student roster, to be retained by the instructor, which lists 
the institution’s code number for each student, the student’s 
name, and a score assigned to the writing sample collected for 
that student. 

4. A student roster, to be returned to ETS later in the study, which 
lists the institution’s code number for each student, a Law School 
Admission Services (LSAS) code number, and the score assigned to 
the writing sample collected for that student. The LSAS code 
numbers will be entered on this roster *by the instructor from a 
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listing of students attending the law school sent to instructors 
by LSAS . This procedure protects the student’s anonymity while at 
the same time allowing for a match with data on file at LSAS. 

I recommend the following steps. First, distribute the statement 
informing students of the study. Second, decide which student writing samples 
should be selected to provide a good range of the quality of writing done by 
students at your institution. Third, get clean copies made of the 20 writing 
samples without any instructor annotations on them. Fourth, read and comment 
on the papers in some detail (preferably using a tape recorder). Fifth, 
complete the first student roster (which you will retain) listing each 
student’s name and the score assigned to their writing sample. Finally, send 
the clean copies, the annotated copies, and the tapes to ETS by January 10, 
1992. 



In February, we will send you clean copies of 20 writing samples from 
another institution. We will request that you provide scores on these in the 
range 6 (high) to 1 (low) as if they had been written by students at your 
institution. No commentaries are needed for these writing samples; only a 
global score is needed. At about the same time, we will ask LSAS to send you 
an alphabetical listing of students attending your law school with LSAS code 
numbers. You will use this listing to enter the LSAS code numbers on the 
second student roster (to be returned to ETS). 

Our Advisory Committee is excited about the project and believes that it 
will have an important influence on legal writing instruction. Your 
assistance in the study is very much appreciated. 

If you have any questions, please give me a call (609-734-5187) or send 
a FAX (609-734-5010). 



Sincerely, 



Hunter M. Breland 
Senior Research Scientist 
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EDUC AT IONAL TESTING SERVICE 




PRINCETON. IS.J. 0 8 541 



600-027 -0000 
CA3LZ-EDUCTZSTSVC 



PROGRAM RESEARCH OFFICE 



February 25,1992 



MEMO TO: Legal Writing Study Participants 



FROM: Hunter M. Breland 



As promised, I am sending you samples of legal writing assignments and 
student responses from another of our participating institutions. We have 
tried, where possible, to disguise the identity of the institution, but in 
some cases it is very difficult. Don’t be fooled, however, by the state in 
which the assignment occurs. Some of our participating institutions have 
assignments related to situations outside their own state. 

We do not have copies of all the statutes and case law relevant to the 
assignments, but we believe that you can determine the essence of the law from 
the better samples. Please let me know if you disagree. 

You may keep the writing samples, so if you want to make notes on them 
as you read them that is okay. I am enclosing rating forms for each of the 
writing samples and request that you enter, at the top, a global score on a 6 
(high) to 1 (low) scale similar to what you did for your own samples. You may 
find it difficult, however, to use the entire range of scores because the 
institution from which the samples came may be very different from your own 
institution. But try to avoid extremes, such as using only two or three score 
levels. And try not to be overly critical because an institution uses a 
format different from your own. 

Once you have entered the global score, we will appreciate your 
indicating what characteristics of a sample you found to be especially weak (- 
-), weak (-), strong (+) , or especially strong (++) by checking that 
characteristic in the appropriate place. There is no need to rate all of the 
characteristics, but checks on a minimum of 10 or so characteristics will be 
helpful. If possible try to rate subelements rather than major categories 
(e.g. "Writing"). These rating forms were developed from the detailed 
commentaries you and other participants sent earlier, and your ratings of them 
will help to determine what aspects of legal writing are deemed most important 
in the 12 participating institutions taken collectively. We will not be 
making institutional comparisons, and your ratings will be kept confidential. 

I will appreciate your comments on the form and the draft taxonomy of legal 
writing characteristics enclosed. 
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We will appreciate receiving these completed rating forms no later than 
March 31 so that we can initiate the paperwork to have your final payment of 
$200 (plus expenses) sent before the academic year is over. Make sure that 
you send a statement of all expenses incurred (with receipts) along with the 
rating forms. Please inform me also if you have not received the first 
payment of $300 (plus expenses). 

Many thanks for your efforts to date, including dealing with the ETS 
bureaucracy. Please call me if you have any questions. 
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PRINCETON. N.J. 08541 



EDUCATIONAL TESTING SERVICE 



609-92 7 -gooo 
CASLE-ZDUCTESTSVC 

program research office 



July 23, 1992 



Dear [Legal Writing Professor or Instructor]: 

You indicated earlier that you would be interested in rating more 
writing samples from another law school. At its last meeting, the Advisory 
Committee decided that another set of ratings would be useful. Enclosed is a 
set of 20 writing samples different from the one you rated before. The same 
basic procedure is to be followed, although we have made some minor 
modifications in the rating form. One difference, however, is that we would 
like the samples returned this time (without marks or comments written on 
them). We would like them returned because of the possibility (though 
unlikely) that still a third rating might be desired. 

I am enclosing rating forms for each of the writing samples and request 
that you enter, at the top, a global score on a 6 (high) to 1 (low) scale 
similar to what you did before. Please attempt to use the entire range of 
scores; the previous ratings indicate that this is possible within all of the 
institutional sets. Note that the ID codes entered on the forms refer to 
institution and student within institution. Thus "11-1" identifies 
Institution #11 and Student #1, "11-2" identifies Institution #11 and Student 
#2, etc. 

Once you have entered the global score, we will appreciate your indicat- 
ing what characteristics of a sample you found to be especially weak ( — ) , 
weak (-) , strong (+) , or especially strong (++) by checking that 
characteristic in the appropriate place. There is no need to rate all of the 
characteristics, but checks on a minimum of 10 or so characteristics will be 
helpful. If possible, try to rate sub-elements rather than major categories 
such as "Writing" or "Analysis." We will not be making institutional 
comparisons, and your ratings will be kept confidential. I will appreciate as 
well your comments on the form and the draft taxonomy of legal writing 
characteristics enclosed. 

A few comments we have received from previous raters may be helpful. 

Some raters felt the need for a neutral rating, that is, neither positive nor 
negative. We assume that an element is neutral if a rater makes no checks for 
it. Therefore, if you feel neutral about an element, just leave it blank. 
Another comment we received indicated the need for a "Not Applicable" rating 
to cover situations where an institution may not have required an element of 
the rating form (Headings, for example). After some thought, we decided not 
to introduce such a rating because, if raters believe an element is important, 
then we would want the analyses to show that — even if some institutions do not 
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require these elements. Finally, some raters have wanted to make finer 
distinctions than we are requesting — for example, a rating between (-) and 
( — ) or between (+) and (++) . Because most raters do not want to make these 
fine distinctions, we have decided to leave the form as it is. 

We will appreciate receiving these completed rating forms no later than 
October 1. Payment of $200 (plus expenses) will be sent to you when the 
ratings and the samples are returned to us. You may receive an additional 
contract letter from our Contracts Office. That will have to be signed and 
returned before the payment will be released. 

Many thanks. Please call if you have any questions. 



Sincerely, 



Hunter M. Breland 
Senior Research Scientist 
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ID. 



First Draft of Rating Fora 
Overall Score 



Questions Presented 

Issue and key fact identification 

Legal expression and context 

Brief Answer 

Conciseness 

Relationship to Questions Presented (On point) 

Completeness 

Statement of Facts 

Relevance of facts stated 

Recognition of legally significant facts 

Adequacy of detail 

Objectivity (seeing both sides) 

Discussion 

Issue and sub-issue description 

Research 

Authority case selection and relevance 

Authority case description and understanding . 

Identification of controlling authority 

Analysis 

Application of law to facts 

Application of rationale 

Use of key facts 

Analogy and comparison of facts . . 

Treatment of distinctions and counterarguments . . . . 

Reasoning 

Support of statements 

Completeness of explanation and discussion . . . . . 

Focus 

Conclusion 

Reasons 

Responsiveness to Questions Presented 

Comprehensiveness and thoroughness . . 

Writing 

Discourse . . . 

Introductions and thesis statements 

Organization 

Paragraphing, transition, flow, logical continuity. . 

Paraphrasing 

Style and tone 

Diction (word choice, precision) 

Conciseness (non-verbosity) 

Coherence and clarity 

Sentence sense and simplicity 

Use of parallel structure 

Mechanics 

Citations (use and form) 

Quotations (use and form) 

Editing and proofreading 

Subj ect/verb agreement 

Pronoun usage 

Punctuation 

Spelling 

Responsiveness to Assignment Directions 
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First Revision of Rating Fora 



ID. 



Overall Score, 



Questions Presented 

Issue and key fact identification 

Legal expression and context 

Brief Answer 

Conciseness 

Relationship to Questions Presented (on point) 

Statement of Facts 

Relevance of facts stated 

Accuracy 

Adequacy of detail 

Objectivity 

Discussion . i 

Issue and sub-issue description 

Research 

Authority selection and relevance 

Authority description and understanding 

Rule identification . . 

Analysis 

Application of law to facts . . . 

Application of rationale 

Use of key facts 

Analogy and comparison of facts 

Treatment of distinctions and counterarguments . . . . 

Reasoning 

Support for statements . . . 

Completeness of explanation and discussion . . 

Focus . . ...... 

Conclusion 

Reasons 

Responsiveness to questions presented 

Compr ehens ivene s s and thoroughness 

Writing 

Organization 

Introductions and thesis statements . . 

Transition, flow, logical continuity, coherence . . . 

Paragraphing 

Use of headings and sub-headings 

Style 

Clarity 

Conciseness .... 

Sentence sense and simplicity 

Diction (word choice, precision) 

Tone and attitude 

Paraphrasing and use of quotations 

Mechanics 

Citations (use and form) 

Editing and proofreading 

Grammar and usage 

Punctuation 

Spelling 

Responsiveness to Assignment Directions 
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Second Revision of Rating Form 



ID 



Overall Score. 



Questions Presented 

Issue and key fact identification 

Legal expression and context 

Brief Answer 

Conciseness 

Relationship to Questions Presented (on point) 

Statement of Facts 

Relevance of facts stated 

Accuracy 

Adequacy of detail 

Objectivity 

Discussion . . 

Issue and sub-issue description 

Research 

Authority selection and relevance 

Authority description and understanding 

Rule identification 

Analysis 

Application of law to facts 

Application of rationale 

Use of key facts 

Analogy and comparison of facts 

Treatment of distinctions and counterarguments . . . . 

Reasoning 

Support for statements 

Completeness of explanation and discussion 

Focus 

Conclusion 

Reasons 

Responsiveness to questions presented 

Comprehensiveness and thoroughness 

Recommendations for further action 

Writing 

Organization 

Choice of organizational strategy 

Introductions and thesis statements 

Transition, flow, logical continuity, coherence . . . 

Paragraphing 

Use of headings and sub-headings 

Style 

Clarity 

Conciseness 

Sentence control 

Diction (word choice, precision) .... 

Tone and attitude 

Paraphrasing and use of quotations 

Mechanics 

Citations (use and form) 

Editing and proofreading 

Grammar and usage 

Punctuation 

Spelling 

Responsiveness to Assignment Directions 
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Appendix D. An Exemplary Student Legal Memorandum 
With Instructor’s Assignment and 
Commentary 
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Assignment 3 

Beth Casey was a police officer for the City of Ship 
Bottom, New Jersey, ih^pctober, 1991 she was assigned to 
Beat 24, the only beat in which two officers rode together 
in a squad car. Her partner during this rotation was Nancy 
Lagnee While they were on Beat 24, Casey and Lagnee and 
were on the 7 to 3 shift. They were each entitled to 1/2 
hour lunch, which had to be taken between 12:00 noon and 
1:00 p.m. 

On October 15, 1991, Casey was returning from lunch to 
the Ocean Street Station to pick up Lagnee. Lagnee had 
stayed at the station : during the lunch break to tend to some 
administrative matters, and Casey had taken the squad car to 
grab a bite. Regulations of the Ship Bottom Police 
Department permitted officers to drive the Department's 
vehicles to and from lunch and between the station and their 
homes before they came on and after they went off duty. 

Casey was about five minutes away from the station and 
still had about 15 minutes left on her lunch hour when she 
heard over the squad car radio a call coming into the Ocean 
Street Station dispatcher. The call was from the bartender 
at the Broken Bottle Tavern, reporting that he had just been 
beaten and robbed of all the money in the cash register by a 
customer who had evidently had too much to drink. The 
bartender reported that the customer had sped off in a 
silver Porsche with a New Jersey license IM GREAT. 




- 1 - 



72 



When Casey heard the dispatcher take this call she was 
just a block and a half away from the Broken Bottle Tavern. 

As she picked up the radio to inform the dispatcher that she 
was in the imm ediate vicinity, a silver Porsche with a New 
Jersey license IM GREAT spun around the corner and started 
down the street immediately in front of her. As it headed 
down the street, it was going 10 m.p.h over the posted 30 
m.p.h. speed limit. The neighborhood was a combined 
residential and small commercial area of town. Casey turned 
on her siren, flashed the light on the squad car and 
informed the dispatcher that she had the suspect in sight. 

When Casey turned on her siren and flashing light, the 
silver Porsche accelerated. Soon both cars were outside the 
residential area of town and on a two lane flat road that 
ran along the coast. The day was clear, and the pavement was 
dry. Casey was going 60 m.p.h as she passed a sign with a 
posted 45 m.p.h. speed limit. Suddenly, in an attempt to 
escape, the driver of the silver car braked to make a left 
turn. His car "fish tailed" in front of Casey's, bounced 
off a tree on the side of the road and spun back onto the 
road. Casey attempted to brake, but her car careened into 
the Porsche. 

The paramedics arrived at the scene and had to remove 
both drivers from the cars. The driver of the Porsche was 
basically unhurt, except for a few cuts and bruises. Casey, 
however, was severely injured. She had to be hospitalized 
indefinitely and the doctors predicted that she probably 
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would be unable to return to her job on the Ship Bottom 
Police force. 

Casey was entitled to receive Worker's Compensation for 
her injuries. But her bills are exorbitant, and she is in a 
lot of pain. You axe working as an associate in a local law 
firm and she has sought your firm's advice. A partner in 
your firm has found out that the driver of the Porsche was 
Chris Spel lman , a brash young man of about 30. He was one 
of Ship Bottom's wealthiest citizens, having made a lot of 
money in wheeling and dealing in property along Ship 
Bottom's extensive coastline. The day Casey was injured, 
Spellman had had a bit too much to drink at the Broken 
Bottle Tavern, though during the busy lunch hour, the 
bartender was not aware of exactly how much alcohol Spellman 
had consumed. He had assaulted and robbed the bartender on 
a dare from one of his drinking buddies. 

Before you were put to work on the case, your firm had 
decided to sue Spellman and the attorneys had filed a 
complaint charging him with negligence. Spellman's 
attorneys moved for summary judgment, claiming that the 
"fireman's rule" bars Casey from recovering anything from 
Spellman. Your firm must respond to the motion for summary 
judgment. You have been asked to do a memorandum analyzing 
whether you can defeat the motion and get to trial. DO NOT 
discuss the New Jersey Dram Shop Act. This memorandum 
should be 10 to 12 pages long. It is due on Tuesday 
November 26. 



O 
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memorandum 



TO: 

FROM: 

DATE: November 21, 1991 

re- Motion for summary judgment in Casey v. Spellman 



O 

ERIC 



75 



STATEMENT OF FACTS 



Our client Beth Casey, a police officer, sued Chris Spellman 
for injuries she sustained in a collision between their vehicles. 
The accident occurred while Casey was attempting to arrest 
Spellman. Spellman has moved for summary judgment, claiming that 
the fireman's rule protects him from liability. You have asked me 
to determine whether we can defeat his motion. 

Casey worked for the City of Ship Bottom, New Jersey. In 
October 1991, she was assigned to Beat 24, a 7:00 to 3:00 shift 
with 30 minutes for lunch. On Beat 24, two officers ride together 
in the squad car. Casey had just finished eating lunch while her 
partner Lagnee took care of some work at the station. She had 
about 15 minutes left on her break and was returning to the 
station to pick up Lagnee. When she was about five minutes from 
the station, she heard over her squad car radio a call to the 
dispatcher from the bartender at the Broken Bottle Tavern. The 
bartender reported that he had just been beaten and robbed of the 
money in the tavern's cash register by a customer who had 
apparently had too .much to drink. The customer had raced off in a 
silver Porsche with New Jersey tags IM GREAT. When she heard the 
call, Casey was a block and a half away from the Broken Bottle. 
As she picked up the radio to tell the dispatcher that she was in 
the vicinity, a silver Porsche with license IM GREAT rounded the 
corner and started down the street ahead of her. It was going 10 
m.p.h. over the 30 m.p.h. speed limit. Casey told the dispatcher 
she was in sight of the suspect and turned on the squad car's 
siren and flashing light. 
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When Casey switched on her siren and flasher, the driver of 
the Porsche accelerated, and Casey pursued. They soon left the 
town on a flat two-lane road. The posted speed limit was 4 5 
m.p.h. Casey noticed that she was going 60 m.p.h. in pursuit of 
the Porsche. Suddenly, the driver of the Porsche braked and tried 
to make an evasive left turn . His car fishtailed, hit a tree on 
the side of the road, and ricocheted back onto the roadway. Casey 
could not stop in time and her car smashed into the Porsche. The 
driver of the Porsche, Chris Spellman, sustained only minor cuts 
and bruises, but Casey was severely injured. She will probably 
not be able to return to her job with the police force. Casey 
received worker's compensation payments, but her bills exceed the 
allotted amount and she is suffering a great deal of pain. 

On the day of these events , it appeared that Spellman had had 
a bit too much to drink at the tavern, although the bartender did 
not know exactly how many drinks he had consumed. He assaulted 
and robbed the bartender on a dare from one of his drinking 

buddies . 

If Spellman's, motion for summary judgment is granted, our 
client will have to settle for her worker's compensation. If the 
motion is denied, we have a good chance of succeeding at trial or 
settling out of court, and our client would recover additional 
compensation for her injuries and suffering. 

QUESTION PRESENTED 

Whether a motion for summary judgment based on the fireman's 
rule should be granted in favor of a citizen who was suspected of 
assault and robbery, who had had a bit too much to drink, and who 

ERLC 
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was fleeing from a police officer responding to a call during her 
lunch break, when the officer sues the citizen for injuries she 
sustained while attempting to arrest him. 



CONCLUSION 

The evidence in this case is sufficient for the plaintxff to 
defeat a motion for summary judgment. The fireman's rule is the 
law in New Jersey, but there are three exceptions under which 
recovery is allowed by firefighters and police officers. Two of 
these arguably apply to the present case. 

The first exception arises when the officer is not assuming 
any special risks inherent to her occupation. The pursuit of a 
suspected felon is an emergency situation which police officers 
are expected to handle, and Casey was fully prepared to encounter 
the risks involved. Our case does not fall within this exception. 

A second exception permits a finding of liability when the 
officer's injury is the result of the defendant's subsequent act 
which is independent of the act that occasioned the officer's 
initial response. • While it is arguable that Casey could have 
expected Spellman to flee from her and take dangerous evasive 
action, it is equally arguable that Spellman's evasive acts were 
independent of the robbery which brought Casey to the scene. We 
can defeat summary judgment on this exception because there is a 
triable issue of fact as to whether there was an independent act. 

The third exception allows the officer to recover when the 
act which created the hazard that caused her injury constitutes 
willful or wanton misconduct. A jury could find that Spellman's 
actions to avoid arrest (speeding and attempting to make a 

O 
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dangerous turn) which caused Casey's injuries were intentional 
acts committed without regard for the consequences; that is, that 
they were willful and wanton. 

Since it is a question for the jury whether the facts of our 
case fall under the exceptions to the fireman's rule, the motion 
for summary judgment can be defeated. 

ANALYSIS 

Our client Beth Casey seeks recovery from Chris Spellman for 
injuries she suffered while attempting to arrest him. Spellman 
contends that recovery is barred by the fireman's rule and has 
moved for summary judgment. To defeat summary judgment, we must 
show that it is a triable issue of fact whether our case falls 
under the fireman's rule. 

The fireman's rule protecting an owner or occupier of 
property from liability to a fireman "for negligence with respect 
to the creation of a fire" is firmly established in New Jersey. 
Kranth v. Seller . 157 A.2d 129, 130 (N.J. 1960). It is the very 
essence of a fireman's business to deal with hazardous situations, 
and society compensates firefighters for encountering the risk of 
fires by paying them for their services and by providing worker's 
compensation for their injuries. Id ._ at 131. The rule set out in 
Krauth was extended to cover police officers in Berko v. F£ed a, 
459 A. 2d 663 (N.J. 1983). There the Court held that because 

police officers, like fire fighters, are "paid to confront crises 
and allay dangers created by an uncircumspect citizenry , " they too 
should be covered by the fireman's rule. Idj. at 666. 
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In its basic form, then, the fireman's rule seems to apply in 
our client's case. Casey was a police officer injured in the 
course of her duties and therefore should be barred from recovery. 
Since its decision in Krauth , however, the court has carved out 
three principal exceptions to the basic rule. If a jury could 
reasonably find that our case falls under one of those exceptions, 
the defendant's motion for summary judgment will be denied. 

The first exception is that the fireman's rule does not apply 
to an officer who is injured when she is not assuming any special 
risks inherent in her occupation. Car of f v ♦ _ Liberty , Lumber _ Co 
369 A. 2d 983 (N.J. Super. Ct. App. Div. 1977). In other words, an 
officer can recover when she was injured while faced with a 
situation that a member of the general public would face or an 
emergency situation which she is not equipped to handle. Rosa v, . 
nunlcin' Donuts of Passaic , 583 A.2d 1129 (N.J. 1991); AlessiO — X ^. 
lire & Tee- Inc. . 484 A. 2d 24 (N.J. Super.- Ct. App. Div. 1984). 
In Alessio, for example, the plaintiff was an off-duty policeman 
who was injured when, at the request of a tavern employee, he 
attempted to defuse a confrontation between two groups of 
intoxicated patrons. Because the plaintiff was off duty, alone, 
out of uniform, and without back-up from his department, the court 
held that he was subjected to undue risks and denied the tavern's 
motion for summary judgment. Alessio., 484 A. 2d at 30. Unlike the 
policeman in Alessio . Casey was in uniform and in her squad car, 
was responding to a dispatcher's call, and was in contact with her 
department. It might be argued that she was off duty during her 
lunch break, but this first exception to the fireman's rule rests 
not on whether the officer was on duty but on whether she was 
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equipped to handle the risks she faced. The only possible 
counterargument here is that Casey's partner was not present as 
she normally would be. However, it would be difficult to argue 
that Casey would have responded any differently to the situation 
or suffered less serious injuries had Lagnee been with her. 

Even an officer who is on duty could recover under this 
exce p t £ on to the fireman's rule if injured while involved in a 
non-emergency situation. Rosa # 583 A. 2d at 1134; Chipps v. 
Newmarket Condominiu m Ass'n. 549 A. 2d 66 (N.J. Super. Ct. Law Div. 
1988); Caroff . 369 A. 2d 983. In Caroff # a park ranger hurt 

himself when he slipped while crossing the defendant's negligently 
kept road on the way to his quarters. Although he was on duty at 
the time, the ranger could recover from the property owners, since 
he was on the property like any other citizen, not in the role of 
protector of the public or in response to an emergency. Caroff, 
369 A. 2d at 984-987. Similarly, the court in Rosa observed that a 
if policeman entered a donut store while on duty to purchase 
coffee or a donut, he would be a customer for purposes of the law, 
and if he were injured, "the fireman's rule would not prohibit his 
recovery against the property owner." Rosa # 583 A. 2d at 1134. 
A^sin f our case is not analogous. Casey's injuries resulted from 
her pursuit of a suspected felon — certainly an emergency response 
situation — and she was equipped to handle that situation. Our 
case does not fall under the first exception to the fireman's 
rule, so we cannot defeat summary judgment on that basis. 

A second exception to the fireman's rule arises when the 
injury is the result of a subsequent negligent act or omission 
which is independent of the negligence that occasioned the 



officer's presence. Rosa . 583 A. 2d at 1134; McCarthy v. Ehrens . 
514 A. 2d 864 (N.J. Super. Ct. Law Div. 1986); Wietecha v. 
Peoronard . 510 A. 2d 19 (N.J. 1986); Trainor v. Santana . 432 A. 2d 



7 



23 (N.J. 1981). Unless the acts of the defendant after the 
arrival of the officer are "closely connected" to the reason for 
the officer's presence, they are not covered by the fireman's 
rule. Rosa . 583 A. 2d at 1133-1134. 

In Trainor. for example, a patrolman was attempting to arrest 
the defendant after a car chase. When he leaned into the 
defendant's car, the defendant tried to escape, and the officer's 
arm was momentarily trapped in the door. The officer freed 
himself but was then struck by the car as the defendant drove off. 
The court held that the defendant's additional act of negligence 
would bar application of the fireman's rule. Trainor. 432 A. 2d at 
23-25. In our case, Spellman's accelerating when he saw Casey's 
flashers and his attempted evasive maneuver could be considered 
acts independent of the robbery and assault which occasioned the 
pursuit. If so, these acts would bar application of the fireman's 
rule. . 

Another case whose facts are favorable to us is McCarthy , in 
which police officers were called to a New Jersey Turnpike toll 
plaza to deal with a very drunk driver stopped there. As the 
officers approached the toll plaza, the defendant drove away. The 
officers followed him as he drove down the right shoulder and 
attempted to get him to stop. Suddenly, the defendant turned 
across three lanes of traffic and crashed into the guardrail on 
the left shoulder. The plaintiff's deceased (a policeman) was 
killed by another motorist when he walked back across the highway 
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after checking on the defendant. The evasive acts of the 
defendant were held to be independent acts, and therefore the 
plaintiff's suit was not barred by the fireman's rule. McCarthy , 
514 A. 2d at 865-868. In our case, as in McCarthy , the defendant 
made evasive maneuvers after realizing that the officer was 
pursuing him, and these acts were the proximate cause of the 
officer's injuries. There are two distinctions, however. First, 
while McCarthy was summoned to the toll plaza, from which the 
defendant then fled, Casey was called to pursue a vehicle already 
in motion; there was a difference in what the two officers could 
reasonably expect to encounter. Second, the defendant in McCarthy 
was very drunk, whereas there is no conclusive evidence in our 
case of Spellman's level of intoxication. Still, in ruling on a 
motion for summary judgment the court must view the facts in the 
light most favorable to the plaintiff, so Spellman should be 
considered a drunk driver for the sake of the motion. 

These cases suggest that there is good support for the claim 
that Spellman's acts were independent enough of the original act 
occasioning Casey's presence for us to avoid application of the 
fireman's rule. A counterargument can be found in Rosa , however. 
In that case, the officer was summoned to a store to deal with a 
medical emergency. Therefore he could not recover for injuries 
sustained when he slipped in the store's kitchen while trying to 
move the patient, even though it might have been negligent for the 
store to keep their floor in such a slippery condition. It was 
reasonable for him to expect that he might be required to assist 
in handling the patient under less than ideal conditions. Rosa, 
583 A. 2d at 1133-1134. See also Maryland Casualty Co. v. Heiot, 
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540 A. 2d 920 (N.J. Super. Ct. Law Div. 1988) (firefighter who 

slipped on ice while responding to call for first aid barred by 
fireman's rule from recovering against landowner). Similarly, in 
our case, when Casey responded to a call to arrest a suspect 
escaping by car, she must have recognized the possibility that the 
he might take dangerous evasive actions. Indeed, in the words of 
the Berko court, "one who does not know the risks inherent in a 
high speed chase should not engage in high speed chasing." Berko., 

459 A. 2d at 667. 

Does our case fall under this exception? There are analogies 
to Rosa , a recent Supreme Court case. But McCarthy and Trainoi: 
are closer cases on the facts and they should control. Looking at 
the facts in the light most favorable to the plaintiff, as we must 
in a summary judgment situation, it is clearly a triable issue of 
fact whether Spellman's acts of departing from the tavern and 
fleeing from Casey were independent of the act of robbery that 
occasioned Casey's response. Hence we have a good chance of 
defeating summary judgment under this exception. 

The third exception to the fireman's rule is that the officer 
can recover from a defendant whose willful and wanton misconduct 
created the hazard that caused the injury . Entwistle — v • _ Drave _ s , 
510 A. 2d 1 (N.J. 1986); Mahonev v. Carus Chemical Co.. Inc., 510 
A. 2d 4 (N.J. 1986). Willful and wanton conduct is distinguished 
from negligence on the one hand and intentional wrongdoing on the 
other. Mahonev . 510 A. 2d at 9; Bero v. Reaction Motors D iva./ 181 
A. 2d 487 (N.J. 1962). Unlike negligent acts, willful and wanton 
acts are deliberate and therefore more culpable. And unlike the 
intentional actor, the willful and wanton actor does not intend 
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the harm that results; he need only realize the possibility that 
injury will follow from his intentional act. Mahoney: , 510 A. 2d at 
9; McCarthy . 514 A. 2d at 870. The question in our case is whether 
Spellman's conduct which resulted in Casey's injuries could be 
found by a jury to be willful and wanton. 

In Mahonev . a fireman was injured in fighting a chemical fire 

which started when some potassium permanganate powder packaged in 
fiber-paper drums ignited. It turned out that the manufacturer 
(Carus Chemical Co.) knew that the permanganate was very hazardous 
and especially likely to spontaneously combust when packaged in 
fiber-paper drums. Yet despite its knowledge of the risk, it had 
shipped 100 fiber-paper drums of potassium permanganate to the 
warehouse where the fireman was injured. The fireman sued Carus 
for its willful and wanton misconduct that led to his injuries . 
The court overturned summary judgment for Carus, holding that the 
fireman's rule would not provide immunity for Carus 's acts if a 
jury found that they constituted wanton and willful misconduct. 

at 5-13. In our case, it is at least arguable that Spellman 
knew that his conduct - especially his drinking before driving and 
his sudden attempt to turn off the road — would likely result in 
injury. Evidence that he intended the actual harm that resulted 
is unnecessary. It is enough that a jury could reasonably find 
that his misconduct was not mere inadvertence. He certainly 
intended to turn left and was arguably reckless with regard to the 
consequences of that act. Therefore the question of whether 
Spellman's conduct was willful and wanton should go to a jury. 

More support for this argument is offered by McCarthy. In 
that case the defendant, who was intoxicated, drove erratically 
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down the shoulder of the highway with two flat tires, then crossed 
the highway and smashed into the guardrail on the other side. A 
policeman who had been pursuing the defendant was killed when he 
walked back across the highway after checking on the defendant. 
McCarthy . 514 A. 2d at 866-867. The court held that the 
defendant's conduct could be found by a jury to be willful and 
wanton. Id. at 869. Like the defendant in McCarthy . Spellman had 
been drinking before driving and made a dangerous evasive 
maneuver. The evidence of disregard for risks in our case is not 
as strong as in McCarthy . since it is not certain that Spellman 
was legally drunk. Still, there is a good analogy here. Again, 
there is enough evidence of wanton behavior in our case to defeat 
summary judgment; the question should go to a jury. 

Also in support of our effort to defeat summary judgment is 
the fact that while fleeing from Casey, Spellman violated statutes 
that establish speed limits and prohibit driving under the 
influence of alcohol. N.J. Stat. Ann. §§ 39:4-50, 39:4-98 (West 
1990). A jury might find that this conduct was willful and 
wanton, since it was deliberate (i.e., not negligent) and showed 
disregard for the risks. 

Two cases that could be cited against this argument are 
McMahon v. Chrvssikos . 528 A. 2d 104 (N.J. Super. Ct. Law Div. 
1986) and Brown Truckina Co.. Inc, v . Flexon Industries__Corb^_/ 552 
A. 2d 1026 (N.J. Super. Ct. Law Div. 1988), but both can be 
distinguished from our case. In McMahon , a drunk driver who rear- 
ended the plaintiff was sued for punitive damages. It was held 
that the drunk driving did not constitute willful and wanton 
misconduct unless there were "further aggravating factors." 
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McMahon . 528 A. 2d at 109. However, the analogy here is too weak 
to justify summary judgment in our case. McMahon concerned a 
punitive damages claim, not a fireman's rule claim as in our case. 
In fact, the McMahon court itself distinguished the case from 
fireman's rule cases like McCarthy . Id. at 108. Even if McMahon 
did apply, it could be argued that Spellman's speeding and his 
risky evasive maneuver were aggravating factors, so that his 
conduct would still qualify as willful and wanton. 

In Brown , the plaintiff was injured while fighting a fire on 

the defendant's premises. The defendant had failed to comply with 
a statute requiring proper sprinkler equipment. Nonetheless, the 
plaintiff's suit was barred by the fireman's rule; mere violation 
of the statute was not enough for liability because the 
plaintiff's harm was not the direct result of the violation. 
Brown . 552 A. 2d at 1028-1031. Similarly, Spellman's violation of 
the speed limit was not the immediate cause of Casey's injuries, 
so it could not easily be argued that his speeding was willful and 
wanton. However, there remains a strong possibility that 
Spellman's second act, the sudden turn that was the proximate 
cause of Casey's injuries, could be found to be willful and wanton 
(i.e., reckless with regard to the possibility of injury). 

To defeat summary judgment under this exception, we need only 
show that it is a triable issue of fact whether Spellman's act 
which proximately caused Casey's harm was willful and wanton. The 
possibility that he was intoxicated and the stronger possibility 
that he was aware of the risks involved in his acts of speeding 
and making a sharp turn are enough to send this case to a jury. 
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Instructor's Conunentary 



This is generally a pretty good memo. The mechanical parts 
at the beginning were pretty well done. The large and small 
scale organization was solid. The transitions from one sub-issue 
to the next were clear. The citation form was good, and the 
writing was clear. 

The problems, such as they were, consisted primarily of 
small things missing here and there or some small analytical 
problems. As far as small missing pieces were concerned, the 
conclusion would have been slightly better if you had indicated 
what the "fireman's rule" was before you noted the exceptions. 
Also, in the thesis paragraph, it would have been slightly better 
to identify each of the exceptions, instead of just saying that 
there were three. Finally, when you used the Carof f case in the 
section on assumption of risk, it would have been more candid to 
indicate that it was not a "fireman's rule" case. In fact, 
because it was not, your analysis probably would have been better 
based on Chipps . though you could have noted Carof f with a signal 
and a parenthetical. 

There were a number of subtle analytical points that you 
handled nicely. For example, on the issue using Alessio . it was 
effective to focus on extra risk. The point about extra risk is 
not only what the Alessio case focused on, but it is also 
relevant through Chipps, Celia , and dictum in Rosa . The one flaw 
in your approach, however, is that when you draw some of your 
analysis on the statement in Rosa that the officer would not have 
been barred if he had been in the store as a customer, you are 
treating the dictum as if it were holding. The reasoning would 
have been better and more complete if you had used the holding. 
Then you could have gone on to point out the argument that goes 
the other way and when you are describing that argument, you 
could reinforce some of your reasoning with the dictum from Rosa . 

Throughout the memo you had clear case descriptions, which 
was a strength. In the subsequent act issue, it may have been 
better if you had used Wietecha . rather than Trainor, mainly 
because in Trainor . in contrast to Wietecha . neither the 
appellate court nor the supreme court was perfectly clear about 
what was the original act and what was the subsequent act. 
Although you did a pretty good job with the analogy to Trainor . 
the analysis was somewhat general. The analogy to McCarthy was 
clearer and more specific, because the case itself was more 
specific on what was the act occasioning the officer's presence 
and on what came later. The distinctions that were counter- 
arguments were ok, especially the part about moving vs. stopped. 
The distinction about drinking does not appear to be related. to 
subsequent act. If you were thinking about something specific, 
it did not come through. 
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The counter-argument in subsequent act was ok, but it would 
have been better if you had introduced it with a reason and not a 
case. For example, you could have noted that Spellman could 

argue that the risk of injury in a high speed chase is inherent 

in pursuing a speeding car. You could still use Rosa , but the 
reason you would be using it would be clearer to the reader. The 

quote from Berko on the top of page 9, though it is a good quote, 

is not really helpful to a reader, because the reader does not 
know what the Berko case is about. It would be better to 
describe and apply Berko to bring the quote in, or just to leave 
it out. 

You seemed to have trouble resolving the counter-arguments 
on subsequent act. Given the holdings in the cases, it was too 
facile just to say that there is a jury question. The point 
would work better with the other two exceptions, where the courts 
did, in fact conclude that there was a jury question. But on the 
subsequent act issue, the court's decision on the applicability 
of the exception or not is strictly a matter of law for the 
court. Also do not use rhetorical questions , as you did to 
resolve the counter-arguments on this issue. Use topic sentences 
that tell the reader where you are coming out. 

The willful and wanton issue was well done. You even used 
the holding in Mahoney correctly. The counter-arguments are ok 
here. Note that aside from raising cases as a counter-argument, 
Spellman may also simply argue that he wasn't driving that fast, 
given conditions. 
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Appendix E. Supplementary Tables 



Table El. Descriptive Statistics 



Variable 



Score 1 
Score 2 
Score 3 
Score 4 

Score 2+4 Sum 
Total Score Sum 

Questions Presented 
Issue and key fact ident. 

Legal expression and context 

Brief Answer 
Conciseness 
Relationship to QP 

Statement of Facts 
Relevance of facts stated 
Accuracy 

Adequacy of detail 
Obj ectivity 

Discussion 

Issue and sub-issue descrip. 
Research 

Authority selection 
Authority description 
Rule identification 

Analysis 

Application of law to facts 
Application of rationale 
Use of key facts 
Analogy and comp, of facts 
Treatment of distinctions 
Reasoning 

Support for statements 
Completeness of explanation 
Focus 

Conclusion 

Reasons 

Responsiveness to QP 
Comprehensiveness 



N 


Mean 


S.D. 


Min. 


Max 


237 


3.47 


1.60 


1 


6 


237 


3.39 


1.32 


1 


6 


236 


3.53 


1.45 


1 


6 


237 

237 


3.47 


1.45 


1 


6 


236 


13.78 


4.32 


4 


24 


237 


5.96 


1.44 


2 


10 


237 


6.30 


1.83 


2 . 


10 


237 


5.88 


1.55 


3 


10 


237 


5.98 


1.18 


3 


10 


237 


6.19 


1.55 


2 


10 


237 


6.59 


1.38 


2 


10 


237 


6.39 


1.03 


3 


10 


237 


6.83 


1.26 


3 


10 


237 


6.72 


1.06 


4 


10 


237 


6.57 


1.33 


3 


10 


237 


6.84 


1.12 


4 


10 


237 


6.06 


.62 


4 


9 


237 


6.24 


1.12 


2 


9 


237 


6.35 


.87 


4 


9 


237 


6.32 


1.35 


2 


10 


237 


5.81 


1.52 


2 


10 


237 


6.07 


1.24 


2 


10 


237 


5.98 


.98 


2 


9 


237 


5.92 


1.57 


2 


9 


237 


5.86 


1.18 


2 


9 


237 


6.16 


1.43 


2 


10 


237 


5.62 


1.64 


2 


9 


237 


5.61 


1.35 


2 


9 


237 


5.90 


.86 


3 


8 


237 


5.81 


1.32 


2 


9 


237 


5.41 


1.47 


2 


10 


237 


5.83 


.95 


3 


9 


237 


5.80 


.84 


3 


9 


237 


5.98 


.93 


4 


9 


237 


6.25 


.86 


4 


9 


237 


5.75 


.95 


3 


9 
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Table El, page 2 of 2 



Variable 


N 


Mean 


S.D. 


Min. 


Max 


Writing 


237 


6.03 


.49 


4 


8 


Organization 


237 


6.00 


1.18 


3 


9 


Introductions 


237 


6.01 


1.45 


2 


9 


Transition, flow, etc. 


237 


5.73 


1.50 


2 


9 


Paragraphing 


237 


5.73 


1.34 


2 


9 


Use of headings 


237 


5.68 


1.31 


2 


9 


Style 


237 


6.13 


.79 


4 • 


9 


Clarity 


237 


5.90 


1.44 


2 


9 


Conciseness 


237 


5.78 


1.21 


2 


9 


Sentence control 


237 


5.73 


1.37 


2 


9 


Diction 


237 


5.70 


1.28 


2 


9 


Tone and attitude 


237 


5.99 


1.17 


2 


9 


Paraphrasing/quotations 


237 


5.84 


1.06 


2 


9 


Mechanics 


Citations (use and form) 


237 


5.35 


1.55 


2 


10 


Editing and proofreading 


237 


5.46 


1.30 


2 


9 


Grammar and usage 


237 


5.63 


1.26 


2 


9 


Punctuation 


237 


5.78 


1.16 


2 


9 


Spelling 


237 


6.19 


1.16 


2 


9 


Responsiveness to Assitnment 


237 


6.40 


.82 


4 


8 
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Table E2. Analysis of Two Sets of Instructor Ratings of 
samples from Institutions Other Than Their Own 
(237 cases) 



AVG 



Ouestions Presented 


FRECh 

.48.5 


r i 

.29 


freq 2 

50.6 


r 2 

.30 


FREQ 

49.6 


. 33 


Issue and key fact identification 


. 76.8 


.34 


86. 


1 


.30 


81.4 


.38 


Legal expression and context 


. 57.4 


.22 


70. 


0 


.27 


63.7 


.28 


Brief Answer 


. 29.1 


.10 


46. 


0 


.38 


37.6 


.35 


Conciseness 


. 53.6 


.13 


67. 


5 


.32 


60.6 


.28 


Relationship to Questions Presented 


. 52.7 


.20 


64. 


1 


.26 


58.4 


.31 


Statement of Facts 


.42.2 


.28 


38. 


4 


.35 


40.3 


.30 


Relevance of facts stated 


. 57.8 


.18 


6B. 


8 


.21 


63.3 


.20 


Accuracy 


.44.7 


.17 


52. 


7 


.23 


48.7 


.24 


Adequacy of detail 


. 62.9 


.13 


65. 


0 


.26 


64.0 


.24 


Objectivity 


.48.5 


.12 


46. 


8 


.14 


47.6 


.13 


Discussion 


. 9.3 


.20 


14. 


8 


.29 


12.0 


.32 


Issue and sub-issue description 


. 31.2 


.35 


38. 


8 


.42 


35.0 


.36 


Research 


. 33.3 


.22 


30. 


4 


.40 


31.8 


.40 


Authority selection and relevance 


. 58.2 


.44 


47. 


3 


.38 


52.7 


.51 


Authority description and understanding . . 


. 57.0 


.56 


60. 


3 


.54 


58.6 


.59 


Rule identification 


. 31.6 


.44 


47. 


7 


.39 


39.6 


.49 


Analysis 


. 27.4 


.24 


29. 


1 


.40 


28.2 


.37 


Application of law to facts 


. 58.2 


.60 


67. 


9 


.58 


63.0 


.66 


Application of rationale 


. 35.9 


.45 


38. 


8 


.44 


37.4 


.51 


Use of key facts 


. 52.3 


.52 


56. 


1 


.46 


54.2 


.58 


Analogy and comparison of facts 


. 60.3 


.46 


56. 


5 


.45 


58.4 


.55 


Treatment of distinctions 


.45.1 


.37 


44. 


7 


.32 


44.9 


.37 


Reasoning 


. 22.8 


.32 


25. 


3 


.42 


24.0 


.45 


Support for statements 


. 43.0 


.49 


48. 


,1 


.49 


45.6 


.53 


Completeness of explanation 


. 52.7 


.51 


56. 


,1 


.55 


54.4 


.57 


Focus 


. 21.9 


.33 


27. 


.8 


.42 


24.8 


.48 


Conclusion 


. 27.4 


.21 


27. 


.0 


.30 


27.2 


.29 


Reasons 


. 24.5 


.31 


33. 


.3 


.26 


28.9 


.24 


Responsiveness to questions presented . . . 


. 27.0 


.08 


32. 


.9 


.25 


30.0 


.17 


Comprehensiveness and thoroughness .... 


. 25.3 


.15 


31. 


.2 


.27 


28.2 


.19 


Recommendations for further action .... 


— 


— 


16 


.5 


.11 






Writing 


. 11.4 


.14 


10 


.1 


.23 


10.8 


.27 


Organization 


. 46.8 


.40 


32 


.9 


.41 


46.0 


.54 


Choice of organizational strategy .... 


— 


— 


57 


.4 


.47 






Introductions and thesis statements . . . 


. 53.2 


.46 


59 


.1 


.49 


56.1 


.55 


Transition, flow, and logical continuity. 


. 58.2 


.56 


65 


.4 


.61 


61.8 


.66 


Paragraphing 


.49.8 


.34 


44 


.3 


.32 


47.0 


.38 


Use of headings and sub-headings .... 


. 38.4 


.22 


36 


.3 


.13 


32.7 


.15 


Style 


. 23.2 


.31 


27 


.0 


.35 


25.1 


.44 


Clarity 


. 52.3 


.57 


62 


.3 


.59 


57.3 


.66 


Avoidance of verbosity 


.42.6 


.36 


45 


.6 


.32 


44.1 


.37 


Sentence control 


. 51.9 


.45 


48 


.1 


.39 


50.0 


.45 


Diction (word choice, precision) .... 


.48.1 


.50 


46 


.0 


.35 


47.0 


.45 


Tone and attitude 


. 35.4 


.34 


43 


.9 


.25 


39.6 


.33 


Paraphrasing and use of quotations . . . 


. 29.1 


.26 


31 


.6 


.22 


30.3 


.29 


Mechanics 


. 19.8 


.17 


28 


.3 


.25 


24.0 


.29 


Citations (use and form) 


. 67.5 


.30 


70 


.5 


.31 


69.0 


.32 


Editing and proofreading 


.49.8 


.24 


52 


.7 


.40 


51.2 


.38 


Grammar and usage 


.49.8 


.40 


43 


.5 


.50 


46.6 


.50 


Punctuation 


.40.9 


.30 


44 


.7 


.40 


42.8 


.40 


Spelling 


. 31.6 


.23 


35 


.6 


.31 


33.6 


.27 


ResDonsiveness to Assignment Directions . . . 


. 23.6 


.27 


33 


.3 


.20 


28.4 


.23 



O 
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Table E2 continued: 



Notes : 

FREQ X The percentage of the first set of rating forms having a response 

(positive or negative) for a given taxonomy element. 

FREQ 2 The percentage of the second set of rating forms having a response 

(positive or negative) for a given taxonomy element. 

r x The correlation between the taxonomy element score for the first 

set of rating forms and Score 2. 

r 2 The correlation between the taxonomy element score for the second 

set of rating forms and Score 4. 

r 3 The correlation between the sum of the taxonomy element scores for 

the first and second set of rating forms and the sum of Score 2 
and Score 4. 
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Table E3. Analysis of Sex Differences 

(N — 229, 138 males and 91 females) 



Males Females p 



Variable 


Mean 


S.D. 


Mean 


S.D. 




Score 1 


3.31 


1.58 


3.60 


1.60 


.17 


Score 2 


3.29 


1.33 


3.50 


1.30 


.24 


Score 3 


3.34 


1.48 


3.71 


1.37 


.05 


Score 4 


3.39 


1.44 


3.57 


1.46 


.37 


Score 2+4 Sum 


6.68 


2.29 


7.06 


2.32 


.22 


Total Score Sum 


13.32 


4.36 


14.27 


4.21 


.10 


Ouestions Presented 












Issue and key fact ident. 


6.25 


1.89 


6.34 


1.75 


.71 


Legal expression and context 


5.81 


1.62 


5.90 


1.47 


.67 


Brief Answer 


5.87 


1.25 


6.14 


1.05 


.09 


Conciseness 


6.12 


1.68 


6.24 


1.31 


.55 


Relationship to QP 












Statement of Facts 


6.45 


1.07 


6.30 


.98 


.31 


Relevance of facts stated 


6.70 


1.26 


6.97 


1.27 


.12 


Accuracy 


6.59 


1.12 


6.90 


.97 


.03 


Adequacy of detail 


6.56 


1.35 


6.52 


1.04 


.79 


Objectivity 


6.80 


1.11 


6.89 


1.17 


.58 


Discussion 












Issue and sub-issue descrip. 


6.21 


1.10 


6.29 


1.13 


.62 


Research 


24.23 


3.97 


24.87 


3.61 


.22 


Authority selection 


6.18 


1.42 


6.43 


1.23 


.18 


Authority description 


5.76 


1.53 


5.84 


1.49 


.72 


Rule identification 


6.07 


1.24 


6.04 


1.24 


.86 


Analysis 


57.51 


9.40 


58.86 


9.15 


.28 


Application of law to facts 


5.71 


1.52 


6.20 


1.59 


.02 


Application of rationale 


5.85 


1.18 


5.91 


1.20 


.69 


Use of key facts 


6.09 


1.48 


6.38 


1.33 


.05 


Analogy and comp, of facts 


5.59 


1.67 


5.61 


1.65 


.90 


Treatment of distinctions 


5.56 


1.38 


5.65 


1.34 


.65 


Reasoning 


5.88 


.90 


5.93 


.81 


.67 


Support for statements 


5.75 


1.34 


5.89 


1.31 


.45 


Completeness of explanation 


5.41 


1.56 


5.41 


1.31 


.99 


Focus 


5.79 


.95 


5.86 


.94 


.60 


Conclusion 


5.74 


.92 


5.88 


.73 


.22 


Reasons 


5.99 


.96 


5.96 


.92 


.77 


Responsiveness to QP 


6.22 


.85 


6.29 


.88 


.60 


Comprehens ivenes s 


5.76 


.98 


5.77 


.93 


.95 



O 

ERLC 
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Table E3 , page 2 of 2 



Males Females 

Variable Mean S.D. Mean S.D. 



Writing 



Organization 


20.50 


3.88 


20.70 


4.33 


.72 


Introductions 


5.96 


1.50 


6 . 04 


1.38 


.66 


Transition, flow, etc. 


5.71 


1.48 


5.67 


1.51 


.84 


Paragraphing 


5.64 


1.34 


5.78 


1.36 


.46 


Use of headings 


5.72 


1.25 


5.56 


1.38 


.37 


Style 


23.32 


3.56 


23.66 


3.66 


.48 


Clarity 


5.90 


1.45 


5.87 


1.45 


.88 


Conciseness 


5.66 


1.20 


5.87 


1.18 


.20 


Sentence control 


5.67 


1.35 


5.77 


1.35 


.57 


Diction 


5.69 


1.22 


5.74 


1.39 


.78 


Tone and attitude 


5.83 


1.12 


6.19 


1.17 


.02 


Paraphras ing/quotat ions 


5.81 


1.03 


5.86 


1.04 


.74 


Mechanics 


11.16 


1.98 


11.64 


2.12 


.08 


Citations (use and form) 


5.17 


1.59 


5.57 


1.49 


.05 


Editing and proofreading 


5.30 


1.26 


5.65 


1.31 


.05 


Grammar and usage 


5.48 


1.21 


5.80 


1.28 


.05 


Punctuation 


5.68 


1.14 


5.84 


1.13 


.32 


Spelling 


6.14 


1.08 


6.25 


1.23 


.46 


Selected Mechanics* 


15.95 


3.20 


17.02 


3.33 


.01 


Responsiveness to Assignment 


6.37 


.77 


6.42 


.91 


.67 



^Selected Mechanics was computed as the sum of Citations (use and form) , 
Editing and proofreading, and Grammar and usage. 




96 



Table E4. Descriptive Statistics: Workbench Analyses 

(N- 176) 



Statistic 






Variables* 








KIN 


AUT 


CL 


FLE1 


FLE2 


MEAN 


11.33 


11.11 


11.22 


12.64 


48.23 


S.D. 


1.531 


1.724 


1.118 


1.888 


8.104 


MINIMUM 


7.900 


7.100 


9.000 


8. '600 


26.20 


MAXIMUM 


16.30 


17.20 


13.70 


17.00 


64.30 




NOSNT 


NOWDS 


AVSLEN 


AVWLEN 


NOQST 


MEAN 


147.7 


2 . 818E+03 


19.32 


4.857 


3.176 


S.D. 


43.96 


821.5 


2.933 


1 . 854E-01 


3.968 


MINIMUM 


69.00 


1 . 319E+03 


11.50 


4.510 


0.000 


MAXIMUM 


259.0 


4 . 803E+03 


30.00 


5.270 


21.00 




NOIMP 


NOCWDS 


PCWDS 


AVLENCW 


SHTSENT 


MEAN 


3.068E-01 


1 . 628E+03 


58.14 


6.172 


14.32 


S.D. 


7 . 307E-01 


448.2 


3.078 


2 . 889E-01 


2.934 


MINIMUM 


0.000 


802.0 


50.50 


5.650 


7.000 


MAXIMUM 


6.000 


2.715E+03 


69.10 


6.970 


25.00 




PSHTS 


NOSHTS 


LNGS 


PLNGS 


NOLNGS 


MEAN 


35.80 


52.55 


29.32 


16.18 


23.48 


S.D. 


5.340 


16.74 


2.934 


4.419 


8.675 


MINIMUM 


18.00 


23.00 


22.00 


6.000 


9.000 


MAXIMUM 


49 . 00 


94.00 


40.00 


28.00 


47.00 




LGESTS 


WHERE 


SHTSTS 


WHERE2 


SMPLP 


MEAN 


69.48 


60.91 


2.068 


42.81 


45.36 


S.D. 


19.46 


49.45 


4 . 084E-01 


25.19 


10.22 


MINIMUM 


35.00 


1.000 


1.000 


1.000 


22.00 


MAXIMUM 


132.0 


245.0 


6.000 


191.0 


80.00 




SIMPNO 


CPLEXP 


PLEXNO 


POUNDP 


POUNDNO 


MEAN 


67.11 


36.63 


53.87 


6.034 


9.159 


S.D. 


24.91 


8.072 


18.74 


2.890 


5.488 


MINIMUM 


20.00 


13.00 


12.00 


1.000 


1.000 


MAXIMUM 


146.0 


60.00 


109.0 


18.00 


34.00 




CCP 


CCNO 


VERBT 


TOBEP 


TOBENO 


MEAN 


11.93 


17.60 


324.8 


33.62 


109.9 


S.D. 


5.082 


8.951 


106.9 


6.620 


42.11 


MINIMUM 


1.000 


1.000 


121.0 


14.00 


24.00 


MAXIMUM 


29.00 


43.00 


630.0 


47.00 


248.0 




AUXP 


AUXNO 


INFP 


INFNO 


PASSP 


MEAN 


20.74 


66.94 


16.50 


53.60 


15.66 


S.D. 


4.655 


25.77 


3.903 


22.05 


5.005 


MINIMUM 


8.000 


17.00 


8.000 


14.00 


1.000 


MAXIMUM 


33.00 


165.0 


33.00 


147.0 


29.00 



O 

ERIC 
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Table E4 (continued) 





PASSNO 


PREPP 


PREPNO 


CONJP 


CONJNO 


MEAN 


42.59 


10.83 


304.4 


2.568 


73.25 


S.D. 


18.96 


1.102 


91.23 


5 . 822E-01 


28.47 


MINIMUM 


1.000 


7.800 


119.0 


1.300 


18.00 


MAXIMUM 


114.0 


13.20 


554.0 


4.000 


178.0 




ADVP 


ADVNO 


NOUNP 


NOUNNO 


ADJP 


MEAN 


4.144 


121.1 


28.59 


798.5 


15.87 


S.D. 


9 . 305E-01 


55.24 


2.059 


214.8 


2.840 


MINIMUM 


1.900 


27.00 


24.10 


364.0 


10.80 


MAXIMUM 


6.700 


278.0 


35.50 


1 . 357E+03 


25.60 




ADJNO 


PRONP 


PRONNO 


NOMP 


NOMNO 


MEAN 


438.1 


3.316 


95.06 


3.000 


81.92 


S.D. 


119.2 


9 . 141E-01 


44.55 


8 . 751E-01 


27.54 


MINIMUM 


195.0 


1.300 


24.00 


1.000 


35.00 


MAXIMUM 


783.0 


6.400 


292.0 


5.000 


172.0 




NOUNO 


PRONOP 


POSOP 


ADJOP 


ARTOP 


MEAN 


35.02 


8.557 


2.409 


26.92 


26.70 


S.D. 


14.57 


6.191 


2.566 


10.29 


12.29 


MINIMUM 


10.00 


0.000 


0.000 


3.000 


4.000 


MAXIMUM 


81.00 


33.00 


13.00 


60.00 


70.00 




TOTOP 


BPREPP 


BPREPNO 


BADVP 


B ADVNO 


MEAN 


67.84 


11.21 


16.59 


10.95 


16.99 


S.D. 


8.026 


4.138 


8.034 


6.372 


11.93 


MINIMUM 


48.00 


0.000 


0.000 


1.000 


1.000 


MAXIMUM 


93.00 


23.00 


43.00 


33.00 


61.00 




BVERBP 


BVERBNO 


BSUBCP 


BSUBCNO 


BCONJP 


MEAN 


1.636 


2.295 


6.398 


9.250 


6 . 250E-01 


S.D. 


1.998 


2.797 


3.421 


5.221 


9 . 235E-01 


MINIMUM 


0.000 


0.000 


0.000 


0.000 


0.000 


MAXIMUM 


13.00 


16.00 


17.00 


26.00 


4.000 




BCONJNO 


BEXPP 


BEXPNO 






MEAN 


9 . 375E-01 


1.392 


2.063 






S.D. 


1.486 


1.287 


2.020 






MINIMUM 


0.000 


0.000 


0.000 






MAXIMUM 


7.000 


8.000 


10.00 






*See next page 


for explanation 


of variable 


name abbreviations . 
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Table E4 (continued) 



Variable Name Abbreviations 



Abbreviation Variable Description 



1 . 


KIN 


Kincaid readability grade 


2. 


AUTO 


Auto readability grade 


3. 


CL 


Coleman-Liau readability grade 


4. 


FLE1 


Flesch readability grade 


5. 


FLE2 


Flesch readability index 


6. 


NOSNT 


Number of sentences 


7. 


NOWDS 


Number of words 


8. 


AVSLEN 


Average sentence length 


9. 


AVWLEN 


Average word length 


10. 


NOQST 


Number of questions 


11. 


NOIMP 


Number of imperatives 


12. 


NOCWDS 


Number of content words 


13. 


PCWDS 


Percentage of content words (as percentage of total words) 


14. 


AVLENCW 


Average length of content words 


15. 


SHTSENT 


Short sentence word length 


16. 


PSHTS 


Percentage of short sentences 


17. 


NOSHTS 


Number of short sentences 


18. 


LNGS 


Long sentence word length 


19. 


PLNGS 


Percentage of long sentences 


20. 


NOLNGS 


Number of long sentences 


21. 


LGESTS 


Longest sentence (number of words) 


22. 


WHERE 


Location of longest sentence (sentence number) 


23. 


SHTSTS 


Shortest sentence (number of words) 


24. 


WHERE2 


Location of shortest sentence (sentence number) 


25. 


SMPLP 


Percentage of simple sentences 


26. 


SIMPNO 


Number of simple sentences 


27. 


CPLEXP 


Percentage of complex sentences 


28. 


PLEXNO 


Number of complex sentences 


29. 


POUNDP 


Percentage of compound sentences 


30. 


POUNDNO 


Number of compound sentences 


31. 


CCP 


Percentage of compound-complex sentences 


32. 


CCNO 


Number of compound-complex sentences 


33. 


VERB! 


Total number of verbs 


34. 


TOBEP 


Percentage of "to be" verbs 


35. 


TOBENO 


Number of "to be" verbs 


36. 


AUXP 


Percentage of auxiliary verbs 


37. 


AUXNO 


Number of auxiliary verbs 


38. 


INFP 


Percentage of infinitive verbs 


39. 


INFNO 


Number of infinitive verbs 


40. 


PASSP 


Passive verbs as percent of non- infinitive verbs 


41. 


PASSNO 


Number of passive verbs 


42. 


PREPP 


Percentage of prepositions (as percent of total words) 


43. 


PREPNO 


Number of prepositions 


44. 


CONJP 


Percentage of conjunctions (as percent of total words) 


45. 


CONJNO 


Number of conjunctions 


46. 


ADVP 


Percentage of adverbs (as percent of total words) 



Table E4 (continued) 



Abbreviation Variable Description 



47. 


ADVNO 


48. 


NOUNP 


49. 


NOUNNO 


50. 


ADJP 


51. 


ADJNO 


52. 


PRONP 


53. 


PRONNO 


54. 


NOMP 


55. 


NOMNO 


56. 


NOUNO 


57. 


PRONOP 


58. 


POSOP 


59. 


ADJOP 


60. 


ARTOP 


61. 


TOTOP 


62. 


BPREPP 


63. 


BPREPNO 


64 . 


BADVP 


65. 


BADVNO 


66. 


BVERBP 


67. 


BVERBNO 


68. 


BSUBCP 


69. 


BSUBCNO 


70. 


BCONJP 


71. 


BCONJNO 


72. 


BEXPP 


73. 


BEXPNO 



Number of adverbs 

Percentage of nouns (as percent of total words) 

Number of nouns 

Percentage of adjectives (as percent of total words) 

Number of adjectives 

Percentage of pronouns (as percent of total words) 

Number of pronouns 

Percentage of nominalizations (as percent of total words) 

Number of nominalizations 

Number of sentences beginning with nouns 

Number of sentences beginning with pronouns 

Number of sentences beginning with possessives 

Number of sentences beginning with adjectives 

Number of sentences beginning with articles 

Percentage of sentences beginning with subject openings 

Percentage of sentences beginning with prepositions 

Number of sentences beginning with prepositions 

Percentage of sentences beginning with adverps 

Number of sentences beginning with adverbs 

Percentage of sentences beginning with verbs 

Number of sentences beginning with verbs 

Percentage of sentences beginning with subject-conjunctions 
Number of sentences beginning with subject-conjunctions 
Percentage of sentences beginning with conjunctions 
Number of sentences beginning with conjunctions 
Percentage of sentences beginning with expletives 
Number of sentences beginning with expletives 



Table E5 . Correlations Between Total Global Score 
and Workbench Variables 

(N - 175) 



Variable 


r 


Variable 


r 


Variable 


r 


1. KIN 


.05 


26. 


SIMPNO 


.10 


51. 


ADJNO 


.16 


2. AUT 


.09 


27. 


CPLEXP 


-.01 


52. 


PRONP 


-.06 


3. CL 


.06 


28. 


PLEXNO 


.12 


53. 


PRONNO 


.08 


4. FLE1 


.02 


29. 


POUND P 


.11 


54. 


NOMP 


-.06 


5. FLE2 


-.02 


30. 


POUNDNO 


.14 


55. 


NOMNO 


.13 


6. NOSNT 


.14 


31. 


CCP 


-.01 


56. 


NOUNO 


.09 


7. NOWDS 


.19 


32. 


CCNO 


.07 


57. 


PRONOP 


-.01 


8. AVSLEN 


.07 


33. 


VERBT 


.12 


58. 


POSOP 


.04 


9. AVWLEN 


.05 


34. 


TOBEP 


-.05 


59. 


ADJOP 


.03 


10. NOQST 


-.04 


35. 


TOBENO 


.09 


60. 


ARTOP 


.17 


11. NO IMP 


-.03 


36. 


AUXP 


.13 


61. 


TOTOP 


-.10 


12. NOCWDS 


.18 


37. 


AUXNO 


.15 


62. 


BPREPP 


.12 


13. PCWDS 


-.10 


38. 


INFP 


-.01 


63. 


BPREPNO 


.17 


14. AVLENCW 


.09 


39. 


INFNO 


.08 


64 . 


BADVP 


-.03 


15. SHTSENT 


.07 


40. 


PASSP 


-.04 


65. 


B ADVNO 


.03 


16. PSHTS 


.00 


41. 


PASSNO 


.07 


66. 


BVERBP 


-.03 


17. NOSHTS 


.14 


42. 


PREPP 


.15 


67. 


BVERBNO 


-.01 


18. LNGS 


.07 


43. 


PREPNO 


.25 


68. 


BSUBCP 


.09 


19. PLNGS 


.06 


44. 


CONJP 


.12 


69. 


BSUBCNO 


.20 


20. NOLNGS 


.16 


45. 


CONJNO 


.19 


70. 


BCONJP 


.05 


21. LGESTS 


.01 


46. 


ADVP 


.02 


71. 


BCONJNO 


.01 


22. WHERE 


.15 


47. 


ADVNO 


.10 


72. 


BEXPP 


-.09 


23. SHTSTS 


-.06 


48. 


NOUNP 


.05 


73. 


BEXPNO 


-.03 


24. WHERE2 


.03 


49. 


NOUNNO 


.22 








25. SMPLP 


-.02 


50. 


ADJP 


-.10 









Note: See Table E4 for variable descriptions. 
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